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This  guidebook  is  one  in  a  series  prepared  for  the  Aeronautical  Systems  Division 
(ASD/EN)  of  the  Air  Force  for  Automatic  Test  Equipment  (ATE)  and  trainer  simulator 
software  acquisition.  It  provides  guidance  to  Air  Force  system  acquisition 
personnel  in  the  preparation,  imposition,  and  enforcement  of  software  configuration 
management  requirements  for  ATE  and  Training  simulator  software  through  all  phase 
of  system  acquisition. 

This  guidebook  is  one  of  a  series  intended  to  assist  the  Air  Force  Program  Office 
and  engineering  personnel  in  software  acquisition  engineering  for  automatic  test 
equipment  and  training  simulators.  Titles  of  other  guidebooks  in  the  series  are 
listed  in  the  introduction.  These  guidebooks  will  be  revised  periodically  to 
reflect  changes  in  software  acquisition  policies  and  feedback  from  users. 

This  guidebook  reflects  an  interpretation  of  DOD  directives,  regulations  and 
specifications  which  were  current  at  the  time  of  guidebook  authorship.  Since 
subsequent  changes  to  the  command  media  may  invalidate  such  interpretations  the 
reader  should  also  consult  applicable  government  documents  representing  authorized 
software  acquisition  engineering  processes. 

This  guidebook  contains  alternative  recommendations  concerning  methods  for 
cost-effective  software  acquisition.  The  intent  is  that  the  reader  determine  the 
degree  of  applicability  of  any  alternative  based  on  specific  requirements  of  the 
software  acquisition  with  which  he  is  concerned.  Hence  the  guidebook  should  only  be 
implemented  as  advisory  rather  than  as  mandatory  or  directive  in  nature. 

This  guidebook  was  prepared  by  the  Boeing  Aerospace  Company. 
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Section  1.0  INTRODUCTION 


Software  Configuration  Management  (SCM), 
is  a  unique  and  distinct  discipline 
essential  to  the  development,  delivery 
and  maintenance  of  contractually  com¬ 
pliant  software  products.  Unlike  soft¬ 
ware  in  earlier  systems,  software  today 
is  being  acquired  as  a  product,  rather 
than  as  data.  This  is  being  done  in 
recognition  of  poorly  defined  software 
requirements,  poorly  planned  resources, 
missed  schedules,  poor  documentation, 
excessive  cost  and  post  delivery  prob¬ 
lems.  In  procuring  software  as  a 
product,  provisions  are  made  for  documen¬ 
tation,  review  and  verification  mile¬ 
stones  very  similar  to  those  which  occur 
in  hardware  development.  When  planned 
early  in  the  conceptual  stage  of  system 
acquisition,  SCM  provides  for: 

a.  Establishment  and  maintenance  of 
documentation  systems  for  product  ident¬ 
ification. 

b.  Efficient  methods  of  defining, 
handling  and  tracking  changes  to  soft¬ 
ware  products. 

c.  Verification  and  accounting  for  as- 
built  product  configuration  and  change 
incorporation. 

d.  Periodic  reviews  and  audits  of 
evolving  product  design. 

e.  Verification  of  ultimate  physical 
and  functional  configuration. 

SCM  complements  software  quality  assur¬ 
ance,  and  other  engineering  management 
disciplines  in  providing  for  delivery  of 
quality  software. 

1.1  PURPOSE 

It  is  the  specific  purpose  of  this  guide¬ 
book  to  present  the  general  principles 
of  software  configuration  management 
interpretive  of  government  policy  which 
apply  to  Automatic  Test  Equipment  (ATE) 
and  Training  Simulator  (TS)  system 
acquisitions.  This  guidebook  is  written 


to  provide  insight  into  some  of  the  pit¬ 
falls  encountered  in  configuration  man¬ 
agement  on  these  systems  followed  by 
recommendations  for  circumventing  those 
pitfalls.  It  assists  the  AF  project 
office  in  developing  and  imposing  cost, 
effective  SCM  requirements  in  Requests 
for  Proposal  (RFP)  and  contracting 
documentation.  It  also  assists  system 
acquisition  planners  in  source  selection 
activities,  proposal  evaluations,  and 
contractor  surveys  and  audits.  During 
the  validation  and  full  scale  develop¬ 
ment  phases,  this  guidebook  provides 
guidance  for  the  review  and  approval  of 
contractor's  Configuration  Management 
Plan  (CMP)  and  Computer  Program  Devel¬ 
opment  Plan  (CPDP).  Finally  the  guide¬ 
book  provides  an  overview  of  the  inter¬ 
relationships  of  the  Air  Force,  contrac¬ 
tor  and  system  vendor  throughout  the 
acquisition  process. 

1.2  SCOPE 

This  is  one  of  a  series  of  guidebooks 
related  to  the  Software  Acquisition 
Engineering  (SAE)  process  for  TS  and  ATE 
ground  based  systems.  The  SAE  guidebook 
titles  are  listed  below: 

Software  Cost  Measuring  and  Reporting 
Requirements  Specification 
Contracting  for  Software  Acquisition 
Statement  of  Work  (SOW)  and  Requests 
for  Proposal  (RFP) 

Regulations,  Specification  and 
Standards 

Measuring  and  Reporting  Software 
Status 

Computer  Program  Documentation 
Requirements 

Software  Quality  Assurance 
Verification 

Validation  and  Certification 
Computer  Program  Maintenance 
Software  Configuration  Management 
Reviews  and  Audits 

Management  Reporting  by  the  Software 
Director 
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This  guidebook  covers  generalized  con¬ 
cepts  and  principles  supplemented  by  con¬ 
siderations  unique  to  ATE  and  TS  soft¬ 
ware  configuration  management. 

The  prime  areas  to  be  covered  are: 

a.  Planning  and  organizing  for  SCM 

b.  Software  specifications  and  end 
item  selection 

c.  Baseline  definition  and  change 
control 

d.  Reviews  and  audits 

The  guidebook  describes  the  interaction 
of  the  Air  Force,  contractor  and  system 
vendor  throughout  the  acquisition  pro¬ 
cess.  It  further  discusses  the  comple¬ 
mentary  nature  of  other  contractor  func¬ 
tional  organizations  (i.e.  QA,  Engineer¬ 
ing,  Procurement)  in  the  software  devel¬ 
opment  process. 

1.3  TS  AND  ATE  OVERVIEW 

This  section  provides  a  brief  sketch  of 
TS  and  ATE  system  characteristics, 
including  the  function  of  the  software 
associated  with  each. 

1.3.1  TS  SYSTEM  CHARACTERISTICS 

The  TS  system  is  a  combination  of  a 
specialized  hardware,  computing  equip¬ 
ment,  and  software  designed  to  provide  a 
synthetic  flight  and/or  tactics  environ¬ 
ment  in  which  aircrews  learn,  develop 
and  improve  the  skills  associated  with 
individual  and  coordinated  tasks  in 
specific  mission  situations.  Visual, 
aural,  and/or  motion  systems  may  be 
included.  Figure  1.3-1  depicts  a  typical 
TS  which  employs  digital  processing 
capability. 

The  computer  system,  integral  to  the 
crew  TS,  can  consist  of  one  or  more 
general  purpose  computers.  The  computing 
hardware  operates  with  floating  point 


arithmetic  and  sufficient  bit  capacity 
to  provide  efficient  use  of  a  simulator 
Higher  Order  Language  (HOL).  When  a 
multi-processor/multi-computer  system  is 
used,  it  must  be  designed  such  that  com¬ 
puters  can  operate  simultaneously  and 
are  controlled/synchronized  by  a  single 
program  (supervisor/executive).  The 
executive  directs  program  execution  and 
regulates  priorities. 

The  simulator  (1)  accepts  control  inputs 
from  the  trainee  (via  crew  station  con¬ 
trols)  or  from  the  instructor  operator 
station;  (2)  performs  a  real-time  solu¬ 
tion  of  the  simulator  mathematical 
model;  and  (3)  provides  output  responses 
necessary  to  accurately  represent  the 
static  and  dynamic  behavior  of  the  real 
world  system  (within  specified  tolerance 
and  performance  criteria).  Since  TS  con¬ 
sist  of  interdependent  hardware  and  soft¬ 
ware,  a  joint  hardware/software  develop¬ 
ment  effort  is  required.  As  the  com¬ 
plexity  of  TS  increases,  simulation  soft¬ 
ware  continues  to  grow  in  complexity, 
size,  and  cost.  Software  costs  can  and 
do  exceed  computer  hardware  costs  in 
many  cases.  Therefore,  it  is  imperative 
that  the  simulation  software  acquisition 
engineering  process  be  subjected  to  for¬ 
mal  system  engineering  planning  and 
discipline  to  ensure  cost-effective 
procurement. 

1.3.2  ATE  System  Characteristics 

ATE  is  defined  as  that  ground  support 
system  which  performs  vigorous  system 
tests  with  minimum  manual  intervention. 
ATE  is  used  in  place  of  manual  devices 
because  it  is  more  cost  effective,  pro¬ 
vides  required  repeatability,  or  repair 
of  the  item  being  tested  requires  the 
speed  which  only  an  automatic  tester  can 
achieve. 

Figure  1.3-2  shows  the  typical  compo¬ 
nents  of  an  ATE  system.  Note  that  there 
are  both  hardware  and  software  elements 
involved.  Most  of  the  elements  shown  in 
the  figure  will  be  found  in  the  majority 
of  ATE  systems  although  the  packaging 
and  interface  design  may  vary  between 
specific  systems. 
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The  controls  and  displays  section  con¬ 
sists  of  the  computer  peripheral  devices 
such  as  control  panels,  magnetic  tape 
cassettes  or  disks,  a  cathode  ray  tube 
(CRT),  keyboard,  and  small  printer.  The 
computer  (normally  a  minicomputer),  as 
controlled  by  software,  operates  the 
peripheral  devices;  switches  test  stim¬ 
uli  on  and  off;  and  measures  responses 
of  the  Unit  Under  Test  (UUT)  (comparing 
to  predetermined  values).  The  operator 
maintains  supervisoiy  control  of  the 
testing  process  through  the  peripherals. 
However,  his  interaction  is  usually  mini¬ 
mal  since,  by  definition,  the  automatic 
test  feature  was  selected  in  preference 
to  an  operator-controlled  test  system. 
ATE  is  normally  designed  to  accomodate 
testing  several  different  articles  of 
system  equipment  (normally  one  at  a 
time).  The  maintenance  level  being 
supported  by  the  ATE  is  determined  by 
logistics  systems  analysis.  The  impor¬ 
tance  of  the  software  portion  of  the  ATE 
system  should  not  be  minimized  since 
both  the  application  of  the  test  stimuli 
and  the  measurement  of  the  result  are 
achieved  via  software.  Arbitrary  func¬ 
tion  generation  and  complicated  wave 
analysis  can  also  be  accomplished  by 
software  and  is  becoming  more  prevalent 
in  ATE  systems.  The  cost  of  ATE  software 


is  a  significan  component  of  total  ATE 
costs  and  design  trades  can  be  performed 
to  minimize  ATE  life  cycle  costs. 

1.4  GUIDEBOOK  ORGANIZATION 

The  guidebook  is  organized  as  follows. 
Section  1.0  is  introductory.  Section  2.0 
identifies  government  documents  applic¬ 
able  to  ATE  and  TS  software  configura¬ 
tion  management.  Section  3.0  discusses 
planning  for  configuration  management 
including  resources,  facilities  and 
scheduling  estimations.  Section  4.0 
addresses  product  identification,  the 
position  of  software  components  within 
the  system  component  hierarchy  and  the 
sub-structure  of  software  components. 
Section  5.0  discusses  baseline  defini¬ 
tion  and  management.  Section  6.0 
describes  change  management  including 
change  classification,  approval, 
accounting  and  verification.  Section  7.0 
describes  the  functions  and  management 
of  a  typical  Computer  Program  Library 
(CPL).  Section  8.0  discusses  formal  de¬ 
sign  reviews  and  configuration  audits  of 
the  Computer  Program  Configuration  Item 
(CPCI).  Sections  9.0  through  13.0  con¬ 
tain  a  bibliography,  topic  vs  government 
specification  cross-reference  index, 
glossary  of  terms,  list  of  abbreviations 
and  acronyms  and  a  subject  index. 


Section  2.0  APPLICABLE  DOCUMENTS 


The  following  are  the  major  documents 
which  apply  to  the  area  of  software  con¬ 
figuration  management. 

AFSCM375-7,  Systems  Management,  Con¬ 
figuration  Management  for  Systems, 
Equipment,  Munitions  and  Computer  Pro¬ 
grams 

MIL-STD-480,  Configuration  Control - 
Engineering  Changes,  Deviations  and 
Waivers 

MIL-STD-482,  Configuration  Status 
Accounting,  Data  Elements  and  Related 
Features 

MIL-STD-483,  Configuration  Management 
Practices  for  Systems,  Equipment, 
Munitions  and  Computer  Programs 

MIL-STD-490,  Specification  Practices 

MIL-S-83490,  Specifications,  Types 
and  Forms 

FED-STD-No.  5,  Standard  Guide  for 
Preparation  of  Proposed  Item  Logis¬ 
tics  Data  Records  and  Proposed  Item 
Identifications  by  Government 
Suppliers 


DOD  Directive  5010.19,  Configuration 
Management 

DOD  Directive  5010.21,  Configuration 
Management  Implementation  Guidance 

MIL-STD-499,  Systems  Engineering  Man¬ 
agement 

AFM66-1,  Maintenance  Management 

AFR65-3,  Configuration  Management 

MIL-STD-1521A,  Technical  Reviews  and 
Audits  for  Systems,  Equipments,  and 
Computer  Programs 

MI  L-S-52779  (AD),  Software  Quality 

Assurance  Program  Requirements 

AFR  800-14,  Management  of  Computer 
Resources  in  Systems 

DOD  Directive  5000.29,  Management  of 
Computer  Resources  in  Major  Defense 
Systems 

SSD-Exhibit  61-47B,  Computer  Program 
Subsystem  Development  Milestones 
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Section  3.0  PLANNING  FOR  SOFTWARE  CONFIGURATION  MANAGEMENT 


There  are  several  key  considerations 
which  USAF  and  contractor  personnel 
charged  with  Software  Configuration  Man¬ 
agement  (SCM)  must  consider  early  in  sys¬ 
tem  acquisition  planning.  Although  ATE 
and  TS  software  have  inherent  differ¬ 
ences  in  structure  and  development 
philosophy,  certain  SCM  planning  ele¬ 
ments  are  common  to  both  systems.  Some 
of  these  elements  are: 

a.  definition  of  software  entities 

b.  specification  of  SCM  requirements 

c.  organizing  for  SCM 

d.  planning  for  required  resources 
and  facilities 

The  following  paragraphs  describe  how 
provisions  for  SCM  are  established  in 
the  early  planning  stages  of  software 
development  through  discussion  of  these 
elements. 

3.1  DEFINITION  OF  SOFTWARE  ENTITIES 

DOD  Directive  5000.29,  Management  of  Com¬ 
puter  Resources  for  Major  Defense  Sys¬ 
tems,  in  declaring  that  software  pro¬ 
ducts  must  be  developed  as  configuration 
items,  provides  a  very  definite  approach 
to  SCM.  In  effect,  all  software  entities 
designated  as  computer  program  configura¬ 
tion  items  (CPU's)  must  be  developed 
with  controls  similar  to  hardware.  A 
CPCI  is,  in  general  terms,  a  program  or 
aggregate  of  related  programs  which 
satisfy  an  end  function  and  are  specifi¬ 
cally  designated  by  the  controlling 
agency  for  configuration  management. 
This  definition  does  not  levy  rigorous 
restrictions  on  software  entities 
selected  as  CPCIs. 

CPCI  selection  is  an  important  activity 
critical  not  only  to  successful  configu¬ 
ration  management,  but  to  technical 
soundness  and  manageability.  Considera¬ 
tions  affecting  the  choice  of  CPU's  are 


discussed  further  in  paragraph  4.1.  How¬ 
ever,  it  is  important  that  planners  for 
SCM  (both  USAF  and  contractor)  realize 
that  CPCI  selection  can  create  diffi¬ 
culties  if,  for  example,  too  many  func¬ 
tions  are  integrated  into  one  CPCI.  If 
development  schedules,  language  mixes, 
facilities  etc.  are  not  compatible  among 
the  components  comprising  the  CPCI, 
serious  SCM  problems  are  encountered. 
Accordingly  SCM  planners  should  become 
involved  in  early  system  design  review 
activities  to  insure  the  selection  of 
manageable  CPCIs. 

3.2  SPECIFYING  SCM  REQUIREMENTS 

There  are  three  major  vehicles  within 
the  RFP  for  specifying  SCM  requirements. 

a.  Data  Item  Descriptions  (DIDs)  for 
software  documentation 

b.  Configuration  Management  Plan 
(CMP) 

c.  Computer  Program  Development  Plan 
(CPDP) 

Documentation  requirements  for  ATE  and 
TS  systems  are  thoroughly  discussed  in 
the  "Computer  Program  Documentation 
Requirements"  guidebook.  They  are 
imposed  on  the  contractor  as  DID's  and 
in  most  cases  they  are  standard  in  con¬ 
tent  and  format.  In  some  cases  they  need 
to  be  tailored  to  meet  the  requirements 
and  unique  aspects  of  ATE  and  TS  soft¬ 
ware.  For  example,  the  existing  DID  for 
preparation  of  Test  Requirements  Docu¬ 
ments  (TRD)  (DI-T-3734),  references 
MI L-STD— 1519  which  is  regarded  as  unsuit¬ 
able  in  many  cases  for  ATE  software.  At 
a  recent  joint  services  conference  on 
ATE  (See  Bibliography  ref.  3),  it  was 
concluded  that  MIL-STD-1519  is  too 
stringent  and  expensive  to  comply  with, 
and  hence  has  been  watered  down  in  many 
contracts.  (A  sub-committee  of  this 
conference  recommended  that  a  more  prag¬ 
matic  specification  common  to  the  tri- 
services  be  generated  for  ATE.)  Planners 
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for  SCM  should  therefore  examine  all  of 
the  documentation  requirements  for  soft¬ 
ware  from  system  specifications  through 
Part  II  specifications;  since  these 
specifications  form  a  population  of  docu¬ 
mentation  comprising  software  baselines 
against  which  configuration  control  must 
be  maintained. 

The  second  major  RFP  element  communi¬ 
cating  SCM  requirements,  is  the  CMP 
written  for  the  entire  system  (hardware 
and  software).  This  plan  provides  for 
overall  management  of  the  system  com¬ 
ponents.  SCM  requirements  in  their  funda¬ 
mental  form  are  usually  specified  herein 
as  an  addendum  to  the  basic  plan. 
Details  of  SCM  control  policy  are  then 
expanded  in  the  CPDP. 

AF  planners  should  insure  that  software 
requirements  levied  within  the  CMP  are 
appropriate  for  all  software  products 
involved.  For  example,  in  some  con¬ 
tracts,  a  portion  of  the  hardware  and 
software  may  be  supplied  as  Government 
Furnished  Property  (GFP)  with  existent 
drawings  in  a  certain  format.  This  for¬ 
mat  may  not  be  compatible  with  new  build 
drawings.  For  software,  the  integration 
of  two  or  more  CPC  I  components  of 
differing  configuration  formats  can  pose 
difficult  control  problems.  Altered  item 
drawings  are  usually  required  to  main¬ 
tain  adequate  control.  The  CMP  therefore 
should  address: 

a.  Basic  product  identification  and 
baseline  definitions. 

b.  Drawing  and  documentation  require¬ 
ments. 

c.  Schedules  for  Design  Reviews  and 
Audits. 

d.  Classes  of  Change  Control  and 
Effectivity  Points. 

e.  Change  Accountability  Methods. 

f.  Unique  software  configuration  man¬ 
agement  requirements. 


The  third  major  RFP  element  which  levies 
SCM  requirements  is  the  CPDP.  The  CPDP 
is  prepared  in  accordance  with 
AFR-800-14,  Vol  II,  para.  3-9,  by  the 
contractor  and  submitted  with  his  pro¬ 
posal.  The  CPDP  is  the  detailed  road  map 
for  software  development  and  is  the 
prime  vehicle  for  guiding  SCM  throughout 
the  CPCI  development  process  (see  fig. 
3.2-1).  The  CPDP  should  address  the 
following: 

a.  The  organization,  responsibili¬ 
ties,  and  structure  of  the  group (s)  that 
will  be  designing,  producing,  and 
testing  all  computer  programs. 

b.  The  management  and  technical  con¬ 
trols  that  will  be  used  during  the  devel¬ 
opment,  including  controls  for  ensuring 
that  all  performance  and  design  require¬ 
ments  have  been  implemented. 

c.  The  methodology  for  ensuring  satis¬ 
factory  design  and  testing,  including 
quality  assurance  (QA). 

NOTE 

The  requirements  of  MIL-S-52779, 
software  QA  Program  Requirements  are 
gaining  increasing  prominence  in  new 
weapon  systems  contracts  for  both 
prime  and  support  equipment.  The 
requirements  of  this  specification 
overlap  to  a  large  degree  with  the 
objectives  of  the  CPDP.  Sometimes 
the  requirement  for  a  software  QA 
plan  is  levied.  In  such  cases,  the 
CPDP  and  the  software  QA  plan  should 
be  prepared  in  close  coordination 
because  of  the  inherent  similarity 
of  objectives. 

d.  The  development  schedule  for  each 
computer  program  configuration  item  and 
proposed  program  milestone  review 
points. 

e.  The  procedure  for  monitoring  and 
reporting  the  status  of  computer  program 
development. 
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f.  The  resources  required  to  support 
the  development  and  test  of  computer  pro¬ 
grams.  Special  simulation,  data  reduc¬ 
tion,  or  utility  tools  that  are  planned 
for  use  in  the  development  of  computer 
programs  should  be  identified. 

g.  The  general  procedures  for  report¬ 
ing,  monitoring,  and  resolving  computer 
program  errors  and  deficiencies  during 
development  and  testing. 

h.  The  methods  and  procedures  for 
collecting,  analyzing,  monitoring,  and 
reporting  on  the  timing  of  time-critical 
computer  programs. 

i.  The  management  of  computer  program 
development  masters,  data  bases,  and 
associated  documentation  including  its 
relationship  to  the  CMP. 

j.  Guidelines  and  checkpoints  for 
ensuring  future  computer  program  growth, 
modularity,  and  ease  of  modification. 

k.  The  approach  for  developing  com¬ 
puter  program  documentation. 

l.  Training  requirements  and  asso¬ 
ciated  equipment  for  the  deployment 
phase. 

m.  Engineering  practices  to  include: 
standards,  conventions,  procedures,  and 
rules  for  program  design;  program  struc¬ 
tures  and  conventions;  display  and  logic 
standards;  input/output  signal  stan¬ 
dards;  and  other  disciplines  affecting 
development. 

n.  Security  controls  and  require¬ 
ments. 

o.  Simulation  techniques  and  tasks. 

A  good  CPDP  should  be  prepared  in  the 
following  format: 

I.  Introductory  description  of  policy 
and  objectives 


II.  Scope  -  definition  of  software 
items  under  CPDP  control 

III.  Description  of  general  procedures 
and  techniques  applicable  to  all 
phases  of  development 

IV.  Narrative  descriptions  of  opera¬ 
tions,  control  methods,  documenta¬ 
tion  products  objectives  and 
review  milestones  by  phase  of  soft¬ 
ware  development.  Phases  to  be 
addressed  are 

(1)  Analysis 

(2)  Design 

(3)  Code  and  checkout 

(4)  Test  and  Integration 

(5)  Installation 

(6)  Operation  and  support 

Each  subsection  should  describe 
clearly  the  milestones  which 
demarcate  a  change  in  development 
phase,  involvement  of  AF  and  man¬ 
agement  in  status  reviews,  documen¬ 
tation  products  released  at  the 
completion  of  each  phase  and  the 
approvals  required.  See  Figure 
3.2-2. 

V.  Appendices  for  special  procedures, 
definitions,  matrices  etc. 

A  well  written  CPDP  is  difficult  to  pre¬ 
pare  because  it  requires  a  certain 
insight  into  factors  difficult  to  pre¬ 
dict  at  the  time  it  is  written.  In  ATE, 
for  example,  factors  such  as  specifica¬ 
tion  completeness  and  accuracy,  program 
development  station  availability,  human 
resources,  volume  of  changes,  costs, 
schedules,  operating  system  capabili¬ 
ties,  etc.  all  affect  the  contractor's 
ability  to  follow  the  CPDP  as  originally 
written.  If  prepared  with  enough  flexi¬ 
bility  of  method  to  accomodate  these 
unknowns,  while  maintaining  minimum 
requirements  for  disciplined  transition 
from  one  phase  to  the  next,  the  CPDP 
will  prove  to  be  a  valuable  SCM  tool. 
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3.3  ORGANIZATION  AND  RESPONSIBILITIES 


3.4  RESOURCES  AND  FACILITIES 


Another  important  SCM  planning  considera¬ 
tion  is  that  of  organizing  for  SCM.  SCM 
usually  requires  the  time-phased  divi¬ 
sion  of  responsbilltles.  In  order  to 
achieve  efficient  control  during  early 
development  phases,  without  undue  con¬ 
straint,  while  insuring  formal  control 
during  official  validation  or  acceptance 
testing  phases.  Ideally  a  separate  con¬ 
tractor  SCM  function  should  be  estab¬ 
lished  to  oversee  and  manage  software 
configuration  control  throughout  all 
phases.  However,  SCM  depends  on  other 
organizations  at  various  times  to  meet 
specific  SCM  objectives.  For  example, 
during  pre  System  Design  Review  (SDR) 
phases,  systems  engineering  organiza¬ 
tions  must  maintain  preliminary  software 
requirements  documents;  and  test  design 
organizations  must  maintain  Line  Replace¬ 
able  Unit  (LRU)  acceptance  test  proce¬ 
dures  from  which  TRD's  must  be  prepared 
and  maintained.  During  preliminary  pro¬ 
gram  design,  code  and  debug,  software 
engineering  must  maintain  source  pro¬ 
grams,  functional  and  detail  flow  charts 
and  narrative  descriptions  suitable  for 
eventual  incorporation  into  CPCI  Part  II 
or  equivalent  specifications.  Finally, 
during  validation,  QA  is  responsible  for 
assuring  the  software  "as  built  and  as 
tested"  configuration  is  documented  in 
an  accurate,  complete  and  contractually 
compliant  format. 

Interlaced  among  all  of  these  time- 
phased  SCM  responsbilltles,  the  SCM 
organization  itself  must  function  as  a 
central  focal  point  for  release  and  main¬ 
tenance  of  all  forms  of  SCM  materials 
used  during  all  phases.  Such  materials 
are  discussed  In  paragraph  3.4, 
"Resources  and  Facilities."  Organization 
for  SCM  therefore.  Is  summarily  depicted 
in  Figure  3.3-1.  As  part  of  contractor 
project  command  media,  these  time  shared 
responsibilities  should  be  documented 
and  committed  to  by  the  various  func¬ 
tional  organizations. 


SCM  organizations  require  adequate  man¬ 
power  and  physical  facilities  for  stor¬ 
age  and  management  of  large  volumes  of 
documentation,  changes  and  computer 
sensible  media.  Among  the  materials  to 
be  managed  for  each  software  and  Item 
are: 

a.  Source  code  lists 

b.  Symbolic  decks 

c.  Flow  diagrams 

d.  Performance  specifications 

e.  Interface  description  documents 

f.  Compilers,  assemblers,  link 
editors  -  loaders 

g.  Data  base  specifications 

h.  Load  maps 

1.  Object  code  listings 

j.  Utility  software 

k.  Changes  and  associated  records 

l.  Media  conversion/duplication  faci¬ 
lities  and  methods 

m.  Problem  reports/status  charges 

n.  Test  Procedures,  version  descrlp- 
t ion  documents 

o.  Design  review  and  audit  schedules 

Management  of  the  above  Is  typically 
accomplished  by  a  Computer  Program 
Library  (CPL)  function  and  may  be 
staffed  by  SCM  or  QA  (MIL-S-52779  (AD) 
requires  establishment  of  a  CPL). 
Improperly  estimating  required  resources 
will  adversely  impact  SCM  objectives. 
Usually  data  does  not  exist  from  earlier 
projects  to  accurately  estimate  SCM/CPL 
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resource  requirements.  Factors  to 
consider  In  estimating  SCM  requirements 
are: 

a.  Documentation  requirements 

b.  Schedules 

c.  Number  of  CPCI's  or  components 

d.  Quantity  of  anticipated  changes 

e.  Quantity  and  type  of  computer 
media 

f.  Cost  limitations 

g.  Configuration  control  aids/ 
utilities 

h.  Reporting  requirements 


Resource  requirements  should  be  reported 
in  manhours/manmonths  and  physical  facil¬ 
ity  requirements  per  CPCI.  Procedures 
should  be  written  defining  personnel 
responsibilities  within  SCM,  CPI  organi¬ 
zation  and  filing  systems,  CPL  access 
requirements,  problem  reporting  and 
statusing  methods  etc.  If  the  SCM  func¬ 
tion  is  well  planned  and  documented,  it 
will  prove  to  be  an  indispensable 
resource  in  the  management  of  computer 
programs. 

Section  5.0  discusses  how  these  func¬ 
tional  and  physical  software  descrip¬ 
tions  are  managed  through  a  series  of 
customer  and  contractor  controlled  base¬ 
lines. 
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Section  4.0  PRODUCT  IDENTIFICATION 


DOD  Directive  5000.29,  Management  of  Com¬ 
puter  Resources  in  Major  Defense  Sys¬ 
tems,  in  declaring  that  software  shall 
be  treated  as  a  configuration  item,  has 
provided  for  hardware-like  developmental 
controls  for  software  products.  By 
imposing  configuration  controls  early  in 
the  analysis  phase  of  system  acquisi¬ 
tion,  the  customer  assures  himself  not 
only  of  adequate  documentation  of  the 
finished  product,  but  of  receiving  a  pro¬ 
duct  which  will  be  more  likely  to  ful¬ 
fill  its  functional  requirements  with 
minimal  maintenance  due  to  errors  or 
design  shortcomings.  Product  identifica¬ 
tion  therefore  implies  not  only  the  CPCI 
name  and  number,  but  the  following  physi¬ 
cal  and  functional  identifiers. 

a.  Performance  requirements  identifi¬ 
cation. 

b.  Interface  requirements  definition. 

c.  Design  definition  -  design  repre¬ 
sentations,  source  and  object  code  list¬ 
ings  and  flow  diagrams. 

d.  Physical  media  identification. 

e.  Support  software  requirements 
identification. 

f.  Installation  and  operating  and 
maintenance  instructions. 

g.  Test  requirements  and  acceptance 
criteria  definition. 

Unless  all  of  the  above  are  defined  for 
a  given  CPCI,  the  product  is  incom¬ 
pletely  identified.  Through  the  various 
systems  of  software  documentation  avail¬ 
able  today  {see  the  guidebook,  "Computer 
Program  Documentation  Requirements)," 
provisions  are  made  for  documenting  the 
above  product  descriptors. 

Although  Air  Force  contracts  normally 
assign  responsbi lity  for  ATE  and  TS  sys¬ 
tem  acquisition  to  one  contractor, 
actual  management  and  acceptance  is  done 


at  the  configuration  item  (Cl)  level. 
For  software,  the  ordered  set  of  instruc¬ 
tions  and  coded  data,  in  a  form  suitable 
for  insertion  into  a  computer,  designed 
to  accomplish  a  specific  set  of  func¬ 
tional  requirements,  is  known  as  a  Com¬ 
puter  Program  Configuration  Item  (CPCI). 
The  selection  of  CPCI's  for  management 
and  control  purposes  is  a  task  critical 
to  successful  software  development  and 
configuration  management.  AFSCM/AFLCM 
375-7  provides  basic  guidance  for  Cl 
selection.  Individual  selections  however 
must  be  the  end  product  of  technical  and 
administrative  systems  engineering 
analysis.  The  following  section  examines 
factors  affecting  this  selection  for  ATE 
and  TS  CPCI's. 

4.1  SYSTEM  SPECIFICATIONS 

System  Specifications  for  ATE  vs.  TS  are 
as  diverse  in  content  and  structure  as 
the  functions  they  serve.  Specifications 
for  ATE  are  written  in  terms  of  stimuli, 
measurement,  accuracy  and  thruput 
requirements.  TS  requirements  are 
written  in  terms  of  real  time  simulation 
requirements.  These  differences  are 
fully  described  in  the  "Requirements 
Specifications"  guidebook.  They  are  dis¬ 
cussed  here  only  to  provide  a  starting 
point  for  a  discussion  of  baselines  and 
CPCI  selections.  Although  they  are 
diverse  in  functional  objectives, 
certain  considerations  common  to  both 
affect  the  selection  of  CPCI's.  Major 
system  level  factors  which  should  be 
considered  in  CPCI  selection  are: 

a.  Processor  -  Program‘s  designated  to 
operate  in  a  given  computer  should  be 
separate  CPCI's. 

b.  Schedule  -  Programs  designated  for 
delivery  at  different  times  (i.e.  operat¬ 
ing  system  software  is  developed  and 
delivered  prior  to  simulation  or  LRU 
test  programs)  should  be  separate 
CPCI's. 
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c.  Deployment  -  Programs  designated 
for  supporting i  limited  intended  usage 
(i.e.  LRU  test  programs  designated  to 
support  contractor  factory  acceptance 
test  should  be  separate  from  the  same 
program  used  for  intermediate  level  main¬ 
tenance)  should  be  separate  CPCI's. 

d.  Contractor  -  Program  development 
by  separate  contractors  (i.e.  LRU  pro¬ 
grams  -  vs.  -  control /support  programs) 
should  be  separate  CPCI's. 

CPC I  selections  are  made  to  optimize 
technical  and  administrative  control. 
The  totality  of  Cl's  and  CPCI's  satisfy 
the  system  specification  or  system  seg¬ 
ment  specification  for  the  applicable 
ATE  or  TS  system.  It  is  important  that 
configuration  management  planners  parti¬ 
cipate  in  System  Requirement  Reviews 
(SRR)  and  System  Design  Reviews  (SDR)  to 
register  their  concerns  on  requirements 
allocation  to  Cl's  and  CPCI's  from  a  con¬ 
figuration  manageability  standpoint. 
Some  typical  concerns  to  register  are: 

a.  Can  anticipated  change  processing 
flow  times  support  development  schedules 
for  the  CPCI? 

b.  Is  the  classification  of  change 
control  and  its  point  of  effectivity 
clear  and  acceptable  for  each  type  of 
CPCI? 

c.  Are  the  CPCI's  sized  for 
manageability? 

d.  Are  resources  available  to  control 
configuration  of  special  required  sup¬ 
port  utilities?  -  (i.e.  host  computer 
facilities  needed  to  develop,  assemble 
compile,  link  edit  etc.) 

e.  Will  the  distributed  architecture 
nature  of  modern  ATE  software  present 
any  special  configuration  Maintenance 
problems?  (i.e.  programs  operating  in 
micro  and  mini  computers  remote  from  the 
central  processor. ) 


f.  Are  formal  review  and  audit  sched¬ 
ules  realistic  for  all  CPCI's. 

g.  Is  there  clear  distinction  between 
CPCI  and  non  CPCI  products. 

In  order  to  build  manageable  CPCI's,  the 
system  specifications  upon  which  CPCI 
development  specs  are  based  must  be 
complete  and  accurate.  System  engi¬ 
neering  analysis  and  design  trade 
studies  should  form  the  bulk  of  justi¬ 
fication  for  CPCI  selections.  The  fore¬ 
going  has  presented  some  system  level 
considerations  common  to  ATE  and  TS  soft¬ 
ware  and  configuration  management.  The 
following  paragraph  discusses  some  SCM 
considerations  unique  to  ATE  and  TS 
which  affect  CPCI  selections,  and  soft¬ 
ware  specifications. 

4.2  SOFTWARE  SPECIFICATIONS  AND  CPCI 
SELECTIONS 

The  evolution  of  software  specifications 
is  shown  typically  in  Figures  4.2-1  and 
4.2-2  for  ATE  and  TS  respectively.  In 
reality  the  actual  relationships  among 
requirements,  development  and  product 
specifications  is  as  diverse  as  the  sys¬ 
tems  themselves.  It  is  noted  in  the 
example  of  Figure  4.2-1  for  instance, 
that  a  software  system  specification 
exists  to  govern  the  performance  of  four 
CPCI's.  This  ensures  a  uniform  approach 
to  development  of  programs  is  adhered 
to,  regardless  of  the  originator  of  the 
test  requirements.  The  fact  that  this 
relationship  varies  among  contracts  is 
inconsequential  to  SCM.  What  is  essen¬ 
tial.  however,  is  that  program  develop¬ 
ment  specifications  are  based  on  CPCI 
selections  which  are  manageable  and  on 
system  requirements  which  are  complete 
and  unambiguous. 

Figure  4.2-3  is  an  example  of  an  ATE  sys¬ 
tem  configuration  in  which  functional 
capabilities  have  been  assigned  to  vari¬ 
ous  "stations"  to  accomodate  a  wide 
range  of  UUT's  destined  for  multiple 
user  locations.  This  figure  graphically 
represents  all  potential  configurations 
of  all  stations  at  any  time. 
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Figure  4. 2-2  TS  Software  Specification  Relationship 


Any  given  station  configuration  is  com¬ 
prised  of  a: 

a.  Central  Processor  Unit  (CPU) 

b.  Functionally  unique  station 

c.  Interface  Test  Adapter  (ITA) 

d.  UUT  Test  Program 

Software  architecture  may  be  centralized 
(residing  and  executing  primarily  in  the 
CPU)  or  distributed  (executing  in  micro/ 
mini  processors  residing  in  the  station 
or  adapter.  Adapter  circuitry  may  be 
active  or  passive.  LRU  test  programs  may 
or  may  not  be  transportable  among  sta¬ 
tions  and  adapters.  Users  may  be  con¬ 
tractors  or  AF  maintenance  personnel.  It 
is  seen  therefore  that  system  configura¬ 
tion  control  can  become  complex  and 
unmanageable  if  rigorous  methods  of  base¬ 
line  definition  and  change  management 
are  not  instituted.  To  appreciate  how 
test  requirements  allocations  to  CPCI's 
can  affect  SCM,  consider  the  case  shown 
in  Figure  4.2-3. 

To  preserve  system  configuration  common¬ 
ality  among  all  users  (a  design  goal  for 
most  ATE),  any  change  in  test  capability 
required  of  any  one  user/site  demands 
that  the  same  change  be  incorporated  in 
all  stations  of  like  configuration.  As 
the  quantity  of  deployed  stations  of  a 
given  configuration  increases,  the  con¬ 
figuration  control  problem  increases. 
Several  possible  ways  of  accomodating 
such  a  change  are: 

a.  Incorporate  required  change  in  CPU 
or  station  hardware/firmware. 

b.  Incorporate  required  change  into 
the  site  unique  adapter. 

c.  Modify  the  UUT  software. 

d.  Any  combination  of  the  above. 

Such  change  implementation  is  a  complex 
matter.  Frequently  design  trade  studies 
and  system  engineering  analysis  are 


required  to  determine  what  capability 
shall  exist  in  the  central  processor  vs. 
the  station  vs.  the  adapter  vs.  the  soft¬ 
ware.  The  resultant  decision  will  affect 
software  and  maintainability  signifi¬ 
cantly.  Therefore  the  clarity  and  com¬ 
pleteness  of  all  specifications  -  sys¬ 
tem,  Cl  and  CPCI  are  mandatory  to  effec¬ 
tive  SCM. 

4.3  CPC I /COMPONENTS  IDENTIFICATION 

Functional  requirements  of  CPCI's  should 
be  allocated  to  subcomponetry  in  accor¬ 
dance  with  many  of  the  same  technical 
and  administrative  ground  rules  as  apply 
to  system  functional  allocations.  The 
principles  of  structured  programming 
about  which  much  has  been  published  (see 
Bibliography  references  4  and  5),  apply 
more  directly  to  TS  software  than  to 
ATE.  For  ATE  test  programs,  similar 
principles  of  modular  decomposition 
apply.  UUT  test  programs  are  generally  a 
sequentially  executed  collection  of 
individual  tests  with  little  inter¬ 
module  (or  inter  test)  compatibility  to 
worry  about.  For  TS  software  the  devel¬ 
opment  spec  (CPCI  Part  I)  governs  the 
structure  of  the  CPCI  design,  while  the 
applicable  programming  standards  and 
naming  conventions  govern  its  format.  As 
the  subfunctions  of  a  given  CPCI  are 
ascertained  in  some  fashion  according  to 
designated  criteria  for  CPCI  decomposi¬ 
tion,  these  subfunctions  are  allocated 
to  modules  (or  test  numbers  in  the  case 
of  ATE).  The  following  is  an  example  of 
a  TS  software  module  identification 
scheme  designed  to  achieve  maximum  infor¬ 
mation  from  the  module  part  number. 

123-45678-42203 

Module  version  number 

Module  number 

Major  system  level 
revision  number 

CPCI  designator  (i.e. 
audio  simulator) 

System  designator 
(weapon  system  X) 


The  above  15  character  part  number  fully 
identifies  a  given  CPCI  module.  The  mod¬ 
ule  can  be  described  from  this  part  num¬ 
ber  as  "the  3rd  version  of  module  No. 
22,  part  of  the  4th  major  system 
revision  to  CPCI  45678  for  system  123. 

Total  traceability  of  the  module  to  its 
parent  CPCI  and  system  configurations 
can  be  maintained  with  this  identifica¬ 
tion  system.  Each  module  can  be  designed 
and  changed  independently  of  other  mod¬ 
ules  while  maintaining  traceability  to 
the  parent  CPCI  version.  Such  control  is 
necessary  to  distinguish  modules  during 
concurrent  development  of  different  CPCI 
versions.  Each  module  part  number  then 
appears  on  an  indentured  parts  list  for 
the  specific  CPCI. 

A  version  description  document  (VDD) 
(Figure  4.3-1)  is  used  to  group  such 
modules  for  a  given  CPCI  version.  It  is 
seen  from  this  figure  that  a  given  CPCI 
version,  (123-45678-401)  is  configured 
as  a  collection  of 

a.  Functional  modules 

b.  Physical  media 

c.  Support  programs  necessary  to 
assemble,  compile  etc. 

d.  Labelling  instructions 

e.  Acceptance  test  requirements 

f.  Object  program  listings 


Everything  required  to  build,  identify, 
test,  install  and  operate  a  given  CPCI 
version  is  called  out  on  the  VDD  in 
indentured  parts  list  format.  In  the 
example  given,  it  is  seen  how  the  module 
revisions  are  progressively  updated  for 
the  next  version  of  the  CPCI.  Control  Is 
maintained  in  this  way  by  preserving  the 
configuration  of  previous  versions  upon 
which  later  versions  are  built. 

Safeguards  are  therefore  provided 
against  the  "what-did-we-bui ld-last 
time?"  syndrome  frequently  encountered 
in  uncontrolled  software  development. 
Several  part  number  relationships  are 
logically  evident  in  this  scheme. 

a.  CPCI  version  -401  contains  module 
versions  which  are  predominantly  "-01" 
(i.e.  -42201). 

b.  Major  CPCI  revisions  (i.e.  -501 

vs.  -401)  contain  corresponding  major 
modular  redesigns  (5XXXX  series  modules) 

c.  Module  or  component  specification 
dash  numbers  correspond  to  the  module 
numbers  (i.e.  module  -42201)  is 
described  by  design  spec  XXX-XXHX-22) 

As  the  discussion  of  baseline  and  change 
control  mechanisms  evolve  in  subsequent 
sections  of  this  guidebook,  the  value  of 
these  inter-relationships  will  become 
more  evident. 


g.  Operating  instructions 


Section  5.0  BASELINE  MANAGEMENT 


A  baseline,  in  general,  is  a  documented 
technical  description  which  becomes  a 
point  within  a  development  process 
against  which  changes  can  be  proposed, 
evaluated  and  incorporated.  Three  config¬ 
uration  baselines  for  ATE  and  TS  soft¬ 
ware  are  described  in  this  section  as  a 
function  of  the  development  cycle.  Base¬ 
lines  are  employed  throughout  the  soft¬ 
ware  life  cycle  to  ensure  an  orderly 
transition  from  one  major  commitment 
point  to  the  next.  A  configuration 
identification  document  or  a  set  of  such 
documents,  formally  designated  and  fixed 
at  a  specific  time  during  a  CI/CPCI * s 
life  cycle  establishes  a  baseline.  Base¬ 
lines,  plus  approved  changes  from  those 
baselines,  constitute  the  current  config¬ 
uration  identification.  This  section 
will  discuss  the  three  formal  AF  recog¬ 
nized  baselines  for  ATE  and  TS  software, 
as  well  as  internal  contractor  con¬ 
trolled  baselines. 

5.1  TYPES  OF  BASELINES 

The  three  formally  recognized  baselines 
applicable  to  all  Cl  or  CPCI  items  are: 

a.  Functional  Configuration  Baseline. 
The  current  approved  or  conditionally 
approved  technical  documentation  for  a 
configuration  item  as  set  forth  in  speci¬ 
fications  and  documents  which  pre¬ 
scribes:  all  necessary  functional  charac¬ 
teristics;  the  tests  required  to  demon¬ 
strate  achievement  of  specified  func¬ 
tional  characteristics;  the  necessary 
interface  characteristics;  key  func¬ 
tional  characteristics  and  key  lower 
level  CPCI's;  and  design  constraints. 

b.  Allocated  Configuration  Baseline. 
The  current  approved  performance- 
oriented  specifications  governing  the 
development  of  CPU's  that  are  part  of  a 
higher  level  configuration  Item,  in 
which  each  specification:  defines  the 
functional  characteristics  that  are 
allocated  from  those  of  the  higher  level 
configuration  item;  establishes  the 
tests  required  to  demonstrate  achieve¬ 


ment  of  its  allocated  functional  charac¬ 
teristics;  delineates  necessary  inter¬ 
face  requirements  with  other  associated 
CI's/CPU's;  and  establishes  design  con¬ 
straints. 

c.  Product  Configuration  Baseline. 
The  current  approved  technical  documen- 
tation  which:  defines  the  configuration 
of  a  CPCI  during  the  production,  opera¬ 
tion,  maintenance  and  logistic  support 
phases  of  its  life  cycle;  defines  all 
necessary  form,  fit,  and  function  char¬ 
acteristics  of  a  CPCI;  defines  the 
selected  functional  characteristics 
designated  for  acceptance  testing;  and 
defines  the  acceptance  tests. 

These  baselines  are  shown  on  Figure 

5.1- 1  as  external  baselines;  i.e.,  con¬ 
figuration  control  is  external  to  the 
contractor  organization;  changes  are 
subject  to  customer  approval.  Software 
configuration  control  is  normally 
initiated  with  the  allocated  baseline, 
which  is  established  by  USAF  acceptance 
and  approval  of  the  computer  program 
development  specifications  at  SDR;  that 
baseline  is  used  as  the  point  of  refer¬ 
ence  for  formal  configuration  control 
(see  Paragraph  11.6)  until  the  product 
baseline  is  established  upon  customer 
acceptance  of  the  computer  program  pro¬ 
duct  specification  at  Functional  Config¬ 
uration  Audit  (FCA)/Physical  Config¬ 
uration  Audit  (PCA). 

Internal  baselines  (not  subject  to  USAF 
control)  should  be  used  by  the  contrac¬ 
tor  to  control  the  software  design 
during  the  development  of  the  computer 
program  product  specification.  This 
covers  the  period  between  the  allocated 
and  product  baselines  as  shown  on  Figure 

5.1- 1.  Two  such  internal  baselines 
should  be  established  for  the  software 
design.  The  internal  baseline  of  the 
basic  design  for  each  computer  program 
should  be  established  at  the  Preliminary 
Design  Review  (PDR)  by  release  of  a 
partial  draft  of  the  computer  program 
specifications.  The  internal  baseline 
for  the  detailed  (module)  design  should 
be  established  at  the  Critical  Design 
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Review  (CDR)  by  release  of  the  completed 
draft  of  the  computer  program  product 
specifications.  The  internal  baseline 
for  the  detailed  (module)  design  should 
be  established  at  CDR  by  release  of  the 
completed  draft  of  the  computer  program 
product  specification  less  listings. 
Internal  baselines  for  testing  of  the 
computer  program  should  be  established 
by  release  of  a  computer  program  config¬ 
uration  drawing,  release  of  the  source 
code  to  a  CPL,  and  release  of  the  object 
code  to  quality  assurance  for  control 
during  test  conduct. 

The  foregoing  is  a  generalized  descrip¬ 
tion  of  baseline  control  for  all  con¬ 
figuration  items.  It  is  intended  to 
demonstrate  the  three  basic  develop¬ 
mental  milestones  for  configuration  con¬ 
trol.  Because  of  certain  fundamental 
differences  between  ATE  and  TS  software, 
each  must  be  treated  differently  with 
respect  to  baseline  control. 

5.2  BASELINE  CONTROL  VARIATIONS- 
ATE  VS  TS 

The  following  subparagraphs  discuss  some 
of  the  variations  in  baseline  control 
required  to  accommodate  some  inherent 
differences  between  ATE  and  TS  software. 

5.2.1  Intermodule  Dependence 

Perhaps  the  most  significant  difference 
between  ATE  and  TS  software  relative  to 
baseline  control  is  that  simulator  soft¬ 
ware  is  designed  to  simulate  a  real  time 
situation  requiring  modular  interplay, 
whereas  ATE  programs,  (i.e.  UUT  pro¬ 
grams)  execute  test  commands  sequen¬ 
tially  in  a  batch  processing  mode.  TS 
programs,  therefore,  require  both 
"inter-"  and  "intra-"  module  compati¬ 
bility  and  control  documentation. 

Good  design  and  configuration  control  of 
inter-module  interface  signal  processing 
(timing  and  sizing  of  intermodule  data 
traffic)  is  an  essential  requirement  for 
TS  software.  On  the  other  hand,  ATE  soft¬ 
ware,  although  It  too  has  interface  soft¬ 
ware  compatibility  requirements  (among 


control,  support  and  test  components)  is 
comprised  mostly  of  test  programs  which 
are  independent,  sequentially  executed 
test  statements.  This  difference  is 
shown  diagramatically  in  Figure  5.2-1. 
It  is  seen  here  that  during  deployment, 
intermodule  compatibility  control  is  not 
as  much  a  factor  for  ATE  software  while, 
it  is  a  significant  factor  for  TS  soft¬ 
ware. 

Interface  control,  therefore,  must  be 
established  by  interface  control  docu¬ 
mentation  which  defines  detailed  inter¬ 
face  requirements.  This  becomes  a 
portion  of  the  allocated  and  product 
baseline  documentation  in  appropriate 
CPCI  specifications.  Interface  control 
documentation  establishes  specific  func¬ 
tional  or  physical  relationships  that 
must  exist  between  computer  programs  and 
computers,  other  system  equipment  and 
other  CPCI's;  and  between  computer  pro¬ 
gram  components  of  CPCI's  to  achieve 
integration  and  compatibility.  The 
development  and  verification  of  inter¬ 
face  documentation  and  its  inclusion  in 
the  appropriate  specifications  are  essen¬ 
tial  prerequisites  to  the  completion  of 
baseline  definitions. 

5.2.2  Distributed  Processing 

Another  significant  difference  between 
ATE  and  TS  software  which  affects  base¬ 
line  control  is  the  distributed  architec¬ 
ture  nature  of  ATE  software.  The  prolif¬ 
eration  of  the  mini  and  micro  computer 
has  changed  the  structure  of  ATE  system 
software  from  one  characterized  by  cen¬ 
tral  processing  to  one  of  distributed 
processing  (i.e.  more  functions  being 
performed  by  the  remote  station  rather 
than  by  the  central  computer.)  Control 
of  firmware  (programmable  read  only 
memories)  (PROM),  is  becoming  a  config¬ 
uration  control  ingredient  essential  to 
overall  ATf,  software  configuration  con¬ 
trol.  Control  of  software  configuration 
is  now  more  hardware  dependent.  Mini- 
functions  such  as  signal  generation  and 
analysis  are  being  done  by  the  micro¬ 
processor  while  limit  Information  (pulse 
amplitude,  width,  rep  rate)  is  supplied 
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by  the  UUT  test  program.  The  software 
configuration  controller,  therefore  must 
know  what  programs  exists  in  PROMS  as 
well  as  in  the  UUT  source  program  to 
fully  configure  the  UUT  test  software. 

5.2.3  Change  Volume 

Another  significant  factor  affecting 
baseline  control  differences  between  ATE 
and  TS  programs  is  the  volume  of 
changes.  During  developmental  phases, 
change  volume  is  variable  for  both  sys¬ 
tems,  however,  during  deployment  changes 
to  system  LRU's  affect  ATE  software 
differently  than  TS  software.  Once 
deployed,  an  avionics  LRU  change  would 
have  to  be  functionally  significant 
before  it  would  affect  a  simulator  pro¬ 
gram.  It  would  have  to  alter  functional 
capability  of  the  system  before  It  would 
require  a  corresponding  change  to  the 
simulator  program.  For  ATE  software,  on 
the  other  hand,  many  of  the  "as-built" 
changes  require  UUT  program  changes  to 
insure  functionality  has  been  restored 
or  to  accommodate  signal  parameter  limit 
changes. 

These  variants  for  ATE  and  TS  software 
impact  the  methodology  developed  for 
baseline  control.  The  resources,  skills, 
problem  reporting  methods,  and  change 
incorporation  and  verification  methods 
are  different  as  a  result  of  these 
variants.  In  the  following  subsections, 
we  will  examine  some  typical  baseline 
control  mechanisms  for  both  types  of 
software  in  which  impact  of  these 
variants  will  become  evident. 

5.3  BASELINE  CONTROL  MECHANISM 

There  are  two  distinct  elements  or  sub¬ 
tasks  involved  in  maintaining  baseline 
control : 

a.  configuration  definition 

b.  configuration  accountability 

Configuration  definition  is  a  software 
engineering  responsibility  and  is  com¬ 
prised  of  baseline  definition  plus 


change  definition.  Configuration  defini¬ 
tion  is  maintained  by  releasing  engi¬ 
neering  documents  such  as  Part  I,  Part 
II  specifications  and  VDD's,  plus 
approved  changes  thereto. 

Configuration  accountability  is  the  pro¬ 
cess  of  verifying  that  the  applicable 
computer  sensible  media  reflects  the 
approved  baseline  and  that  changes  to 
that  media  are  incorporated  only  in 
authorized  versions  or  at  other 
committed  effect ivity  points. 

Since  ATE  and  TS  software  are  so  diverse 
structurally  the  following  subsections 
address  baseline  control  separately. 

5.3.1  ATE  Software 

ATE  software  baseline  control  begins 
with  the  system  functional  baseline.  The 
functional  baseline  is  established  by 
the  system  specification  for  the  ATE  and 
SDR.  At  this  point,  top  level  system 
(hardware  and  software)  functional 
requirements  are  defined.  The  system 
specification  defines  required  capa¬ 
bility  in  terms  of: 

a.  Number  and  type  of  UUT's  requiring 
test. 

b.  Computational  capacity  as  a  design 
goal. 

c.  Stimuli /measurement  parameters, 
accuracies. 

d.  Reliability,  maintainability  goals 

e.  Diagnostic  vs.  end-to-end  test 
capabilities. 

These  are  functional  requirements  for 
the  ATE  as  a  system,  irrespective  of  the 
selected  ATE  design. 

The  functional  baseline,  in  effect, 
defines  the  job  to  be  done  by  the,  ATE 
for  a  specified  range  of  products  under 
test.  The  population  of  UUT's  categoriza¬ 
tion,  and  various  logistics  support 
requirements  determine  the  system 
requirements. 
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These  system  requirements  are  then 
"allocated”  to  Cl  and  CPCI's.  For 
example,  a  given  ATE  configuration  may 
be  allocated  into  Cl's  and  CPCI's  as 
follows. 

a.  CPU 

b.  Digital  station 

c.  Video  station 

d.  Microwave  station 

e.  Servo  station 

f.  System  software 

(1)  Control 

(2)  Support 

(3)  Test 

g.  Communication/navigation 

h.  Program  development  station 

Each  of  the  above  is  designated  as  a  Cl 
(or  CPC I  for  software),  the  requirements 
for  which  are  defined  in  a  series  of 
development  specifications  to  form  an 
"allocated"  baseline  for  that  CPCI/CI. 
An  outline  for  an  ATE  system  software 
CPCI  development  spec  is  shown  in  Fig. 
5.3-1.  The  software  allocated  baseline 
is  herein  defined.  It  is  prepared  by  the 
ATE  contractor  during  the  analysis  phase 
of  the  ATE  computer  program  development 
cycle  and  approved  by  the  Air  Force  at 
the  CPCI  PDR. 

The  purpose  of  a  development  specifica¬ 
tion  needs  to  be  reviewed  at  this  point. 
A  development  specification  is  prepared 
primarily  as  a  two-way  agreement  between 
the  Air  Force  and  the  development  con¬ 
tractor.  It  is  prepared  independent  of 
the  design  approach.  It  specifies'  what 
the  software  shall  do  (function),  how 
well  it  shall  do  it  (performance)  and 
under  what  conditions  (design  con¬ 
straints).  In  addition,  it  provides  vali¬ 
dation  requirements  that  define  the 
scope  of  the  validation  program.  The 


specification  is  used  as  the  functional 
and  performance  baseline  for  the  contrac¬ 
tor  in  developing  computer  programs  and 
is  also  used  as  the  baseline  on  which 
Air  Force  acceptance  or  rejection  of  the 
computer  program  is  based. 

A  statement  of  computer  program  require¬ 
ments,  that  have  been  approved  by  the 
contractor  and  the  Air  Force,  is  a 
necessary  instrument  for  a  clear  under¬ 
standing  of  what  the  contractor  will  pro¬ 
duce  and  what  the  Air  Force  expects.  The 
three  basic  categories  of  ATE  computer 
programs  present  different  problems  in 
the  need  and  generation  of  development 
specifications.  The  form  and  substance 
of  a  development  specification  may  vary 
for  each  CPCI  depending  on  the  category 
and  the  degree  to  which  it  must  be 
developed.  Since  ATE  software  categories 
are  different,  they  are  often  designed 
as  separate  CPCIs.  That  implies  separate 
specifications,  separate  development 
schedules,  separate  review  schedules  and 
separate  validation  of  requirements  that 
must  all  be  coordinated  and  eventually 
"sold"  as  a  unit.  This  raises  the  possi¬ 
bility  of  an  ATE  software  system  speci¬ 
fication  covering  all  CPCIs,  as  well  as 
the  individual  CPCI  specifications. 
Since  the  CPCIs  can  and  do  have  separate 
development  schedules,  each  CPCI  would 
have  its  own  PDR  and  CDR  schedule;  e.g. , 
normally  the  control  and  support  soft¬ 
ware  development  precedes  that  of  the 
test  software. 

Most  of  the  time,  control  software  and 
support  software  are  purchased  from  an 
ATE  vendor  as  part  of  a  test  set  or 
separately  from  the  computer  manufac¬ 
turer.  Control  software  and  support 
software  are  usually  identified  as 
separate  CPCIs.  When  these  computer  pro¬ 
grams  are  purchased  "off-the-shelf," 
development  specifications  are  not 
required.  Computer  vendor  documentation 
is  required  for  computer  program  main¬ 
tenance  and  for  the  possibility  of 
making  changes  to  the  purchased  computer 
programs.  The  equivalent  of  a  product 
specification  (Part  II,  MIL-STD-483) 
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ATE  SOFTWARE  DEVELOPMENT  SPEC  OUTLINE 
(ALLOCATED  BASELINE) 


Sect.  3.0  (Part  I  per  MIL-STD-483) 
Requirement 


3.0 

Requi rements 

3.1 

Definitions 

3.1.1 

Station  Test  Software 

3.1.2 

Station  Control  Software 

3.1.3 

Station  Support  Software 

3.2.1 

Station  Test  Software 

3. 2. 1.1 

UUT  Test  Software 

3. 2. 1.2 

Confidence  Test  Software 

3. 2. 1.3 

Operational  Assurance/Fault 

Isolation 

3. 2. 1.4 

In  Process  Self  Test 

3. 2. 1.5 

Maintenance  Software 

3.3.1 

Station  Control  Software 

3. 3. 1.1 

Test  Control  Software 

3. 3. 1.2 

Remote  Mode  Test  Control  Software 

3.3.1 .3 

Program  Development  Control 

Software 

3.4.1 

Station  Support  Software 

3. 4. 1.1 

Maintainability 

3. 4. 1.2 

Operating  System 

3. 4. 1.3 

File  System 

3. 4. 1.4 

Loader 

3. 4. 1.5 

Operator  Interface 

3. 4. 1.6 

Batch  Processing 

3. 4. 1.7 

Language  Processors 

3. 4. 1.8 

Program  Development  Aids 

3. 4. 1.9 

Interactive  Editor 

3.4.1.10 

Automatic  Test  Program  Generator 

3.4.1.11 

Configuration  Control  Aids 

3.4.1.12 

File  System  Cataloguing 

3.4.1.13 

Configuration  Verification 

3.4.1.14 

Media  Conversion 

3.4.1.15 

Maintenance  Software 

Figure  5.3- 1.  ATE  Software  Development  Specification  Outline 


should  be  obtained  from  the  vendor.  Pro¬ 
gram  listings  and  source  code  are  almost 
indispensable.  When  significant  addi¬ 
tions  or  changes  must  be  made  to  the  pur¬ 
chased  control  or  support  software,  a 
development  specification  should  be 
written  covering  the  changes  to  be  made, 
and  the  interfaces  required  with  the  pur¬ 
chased  computer  programs  and  the  test 
equipment.  The  existing  computer  program 
(obtained  from  the  vendor)  is  identified 
as  an  interface  and  the  additional  soft¬ 
ware,  treated  as  a  CPCI,  will  be 
designed,  tested,  reviewed  and  con¬ 
trolled  accordingly.  When  control  and 
support  software  are  to  be  totally 
developed  by  the  contractor,  a  complete 
development  specification  is  required. 

There  is  considerable  controversy  within 
the  Air  Force  and  contractors  as  to 
whether  a  development  specification  is 
applicable  for  test  software.  One  point 
of  view  is  that  test  software  is  a  com¬ 
puter  program  and  must  be  developed  as  a 
CPCI.  Therefore,  a  development  specifica¬ 
tion  is  required  for  proper  control  of 
the  development  process.  The  other  point 
of  view  is  that  test  software  is  derived 
from  a  TRD,  which  can  be  most  effi¬ 
ciently  written  directly  in  the  Abbre¬ 
viated  Test  Language  for  All  Systems 
(ATLAS)  language  by  a  UUT  design  engi¬ 
neer,  thereby  bypassing  the  need  for  a 
development  specification  or  at  least 
regarding  the  TRD  as  the  development 
specification. 

The  point  to  remember  is  that  test  soft¬ 
ware  is  dependent  not  only  on  the  TRD, 
but  on  the  ATE  test  set,  the  ITA  and  the 
station  resident  software  as  well.  It  is 
clear,  therefore  that  the  TRD  itself 
written  independent  of  the  ATE,  ATE  soft¬ 
ware  and  ITA,  is  not  an  adequate  sub¬ 
stitute  for  a  development  specification 
and  cannot  provide  the  design  definition 
and  control. 

The  allocated  baseline  therefore  must  be 
defined  by  a  development  specification 
which  addresses. 


a.  Control  software  requirements 

b.  Support  software  requirements 

c.  Test  software  requirements 

(1)  UUT  test  software 

(2)  Self  test  and  calibration 

d.  Design  requirements  (ATLAS 
version(s) 

e.  Quality  Assurance 

(1)  Configuration  Requirements 

(2)  Functional  and  performance 
verification  requirements 

When  approved  at  PDR,  the  allocated  base¬ 
line  is  authority  to  proceed  into  the 
design  phase  of  ATE  software  develop¬ 
ment.  TRD's  are  prepared  in  accordance 
with  the  development  specification  and 
the  appropriate  contractual  specifica¬ 
tions,  such  as  MIL-STD-1519.  However,  it 
was  recently  concluded  by  the  Joint  Ser¬ 
vices  and  Industry  (see  Bibliography 
ref.  3)  that  MIL-STD-1519  is  too 
stringent  and  expensive.  In  practice 
each  branch  of  the  service  has  had  its 
own  interpretation  and  the  current  con¬ 
sensus  is  that  more  pragmatic  specifica¬ 
tion  is  required  for  TRD's. 

From  the  TRD/development  specification, 
work  can  begin  building  a  "product"  base¬ 
line.  The  product  baseline  is  'def  i  ned-by 
the  product  specification  and  defines 
the  as-built  design.  The  product  speci¬ 
fication  consists  of  three  essential 
parts. 

a.  Logic  flow  diagrams 

b.  Support  narrative  descriptions 

c.  Source  program  listings 

Preliminary  drafts  of  the  product  speci¬ 
fication  are  made  available  at  PDR  and 
completed  drafts  available  at  CDR.  This 
specification  is  maintained  under  Class 
II  control  from  PDR  until  it  becomes 
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deliverable  or  at  PCA  at  which  time  it 
is  henceforth  maintained  under  Class  I 
control. 

When  the  control  and  support  software 
position  of  ATE  software  are  purchased 
off  the  shelf  as  is  the  case  in  most  ATE 
systems  available  today,  the  job  of  main¬ 
taining  baseline  control  of  the  software 
boils  down  to  maintaining  control  of  the 
UUT  test  programs  themselves.  The  UUT 
program  is  one  element  of  what  is  called 
a  Test  Program  Set  (TPS).  A  TPS, 
theoretically  is  everything  an  operator 
needs  to  run  a  UUT  test  given  an  ATE  sys¬ 
tem.  A  TPS  consists  of 

a.  A  UUT  object  code  test  program 

b.  Adapter  or  interface  device 

c.  Operator  instructions  consisting 
of  supporting  data  adequate  to  achieve 
self  maintenance 

(1)  Flow  diagrams,  schematics 

(2)  Source  diagrams 

(3)  Interface  diagrams 

(4)  Test  loop  diagrams 

One  of  the  primary  purposes  of  ATE  soft¬ 
ware  configuration  control  is  to  provide 
at  time  of  delivery,  complete  and  accu¬ 
rate  software  configuration  and  usage 
documentation  to  the  user.  Too  fre¬ 
quently,  developers  of  UUT  programs  are 
under  pressure  to  "debug"  and  deliver 
with  the  result  that  the  related  docu¬ 
mentation  suffers.  Care  must  therefore 
be  taken  to  the  SCM  group  to  insure  pro¬ 
gram  version  updates  are  accompanied  by 
updates  to  all  affected  documentations 
required  by  the  product  specification. 
This  includes 

a.  New  versions  of  compilers/ 
interpreters 

b.  Firmware  changes  to  hardware 
design 


c.  Interface  descriptions 

d.  VDD 

e.  Flow  diagram  +  narratives 


f.  Part 
cable) 


specification 


appli- 


Adapter  changes 


Fortunately,  for  ATE  software,  the  lan¬ 
guage  and  operating  system  accomplish 
much  of  the  configuration  control  job 
through  automated  documentation  tech¬ 
niques. 

For  example,  consider  a  UUT  program 
coded  in  ATLAS.  Most  ATE  systems  avail¬ 
able  today  include  a  complete  repertoire 
of  compilation,  file  management,  text 
editing  and  configuration  control  utili¬ 
ties.  The  system  compiler  accepts  ASCII 
coded  source  language  statements  and 
uses  configuration  information  output  by 
the  ATE  and  adapter/interface  (A/I)  pro¬ 
cessors  to  generate  an  object  code  pro¬ 
gram.  The  ATE  configuration  processor 
accepts  a  description  of  the  instrumenta¬ 
tion,  switching  and  method  of  the  pro¬ 
gramming,  then  outputs  files  used  by  the 
A/I  processor  and  the  compiler.  The  A/I 
processor  accepts  a  description  of 
signal  routing  through  the  A/I  between 
the  UUT  and  the  ATE,  and  data  files  out¬ 
put  by  the  ATE  configuration  processor, 
then  outputs  files  used  by  the  compiler. 
If  the  instrumentation,  switching,  pro¬ 
gramming  method,  adapter  or  interface 
changes,  file  changes  must  be  made  to 
the  ATE  and  A/I  configuration  files  so 
that  accurate  and  up-to-date  system  con¬ 
figuration  information  is  used  by  the 
compiler  (see  Bibliography  ref.  6). 

Other  ATE  system  capabilities,  normally 
supplied  as  part  of  the  vendors  operat¬ 
ing  system,  greatly  aid  the  job  of  main¬ 
taining  ATE  software  baseline  control. 
SCM  personnel  should  become  familiar 
with  the  capabilities  of  the  selected 
ATE  system  to  effectively  utilize  the 
systems  capabilities  for  configuration 
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control  objectives.  Additional  automated 
aids  to  configuration  control  are  (see 
Bibliography  ref.  7): 

a.  Automatic  prevention  of  unautho¬ 
rized  updates  to  configuration  con¬ 
trolled  tapes,  e.g.,  by  checking  an 
authorization  number  in  the  update 
request  with  a  corresponding  change 
approval  code  in  the  SCM  Data  Bank. 

b.  Automatic  SCM  Data  Bank  recording 
of  "was-is"  data  during  controlled  up¬ 
dates,  showing  old  and  new  memory 
contents. 

c.  Automatic  generation  of  flow 
charts  from  the  test  program  coding, 
guranteeing  that  the  flow  charts  are  up 
to  dace  with  the  code,  a  determination 
so  time-consuming  and  difficult  by 
visual  comparison,  that  it  seldom  is  per¬ 
formed  and  then  only  on  a  limited 
sampling  basis. 

d.  Automatic  comparison  of  the  con¬ 
figurations  of  the  pre-  and  post- 
validated  test  program,  resolution  of 
differences  with  SCM  Data  Bank  records 
of  approved  changes,  and  isolation  of 
unapproved  changes. 

e.  Automatic  generation  of  integrated 
configuration  lists  and  other  types  of 
configuration  status  accounting  reports. 

f.  Automatic  generation  of  wire 
lists. 

g.  Automatic  documentation  update  and 
change  distribution. 

h.  Automated  revision  tracking  of 
source  and  object  code  files.  The  revi¬ 
sion  code  is  carried  as  a  field  in  the 
file  name  and  automatically  implemented 
by  the  operating  system  when  the  file  is 
accessed  for  editing. 

Utilization  of  these  automated  aids 
together  with  disciplined  manual  change 
processing  methods  will  provide  for  com¬ 
plete  and  accurate  ATE  configuration 
control.  Methods  of  change  processing 
are  discussed  in  Section  6.0. 


5.3.2  TS  Software 

Baseline  control  for  TS  software  origi¬ 
nates  (similar  to  ATE  software)  with  the 
functional  configuration  baseline  for 
the  simulator  system.  As  shown  in  Figure 
5.3-1,  the  TS  system  specification 
defines  the  functions  to  be  performed  by 
the  TS  without  regard  to  implementation, 
(hardware  or  software).  When  approved  by 
the  System  Program  Office  (SPO)  and 
released  in  the  RFP,  the  contractor  pre¬ 
pares  a  technical  proposal  describing 
how  each  functional  requirement  will  be 
met.  Hardware/software  trade  studies 
determine  the  number  of  tasks  "allo¬ 
cated"  to  software. 

Unlike  ATE  software  the  system  functions 
allocated  to  TS  software  are  not  as 
clear  cut.  (Refer  to  the  guidebook  on 
"Requirements  Specifications,"  for  a  dis¬ 
cussion  of  hardware/software  trades.) 
The  functions  allocated  to  software  are 
usually  undetermined  at  the  time  of  RFP 
release.  The  contractor's  proposal 
identifies  the  software  modules,  their 
interfaces,  and  functions  to  be  per¬ 
formed  by  each  module.  In  accordance 
with  DOD  Directive  5000.29  and  AF  Reg. 
800-14,  software  programs  must  be 
managed  as  a  CPCI's  and  consequently  a 
CPCI  development  spec,  (Part  1)  should 
be  prepared  to  form  an  "allocated"  soft¬ 
ware  baseline.  Some  of  the  problems  asso¬ 
ciated  with  TS  software  requirements 
baseline  control  are 

a.  Untimely  requirements  changes 

b.  Excessive  MIL-spec  design/construc¬ 
tion  constraints 

c.  Excessively  high  fidelity  require¬ 
ments 

d.  Excessive  instructor  displays  and 
controls 

e.  Inexact  verification  requirements 

f.  Ambiguity  in  requirements 
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Unlike  ATE  software,  a  requirement  over¬ 
looked,  misinterpreted  or  changed  with¬ 
out  good  reason  causes  significant  cost, 
schedule  and  configuration  control  diffi¬ 
culties.  Most  test  software  changes  can 
be  accommodated  relatively  easy.  A  pro¬ 
cess  which  is  predictable  -  "apply  stimu¬ 
lus  -  measure  -  output"  can  be  easily 
modified  to  accomodate  additional  para¬ 
meters  or  changes  to  their  limits.  TS 
software  changes  on  the  other  hand,  fre¬ 
quently  require  complex  algorithm  devel¬ 
opment,  module,  intermodule  and  system 
level  checkout.  Program  regression  is  a 
very  real  danger.  Every  effort,  there¬ 
fore  should  be  made  to  develop  a  clear, 
complete  and  practical  allocated  base¬ 
line  for  a  TS  system  software. 

Once  the  allocated  baseline  is  released 
and  approved  at  PDR,  work  can  begin 
building  a  product  baseline.  Figure 
5.3-2  depicts  a  typical  simulator  soft¬ 
ware  program  architecture  broken  down 
into  functional  elements. 

This  figure  identifies  functionally 
separate  software  elements  without  con¬ 
sideration  of  interface  relationships  or 
intermodule  dependencies.  These  dependen¬ 
cies  exist,  however,  and  must  be  fully 
defined  in  the  Interface  Design  Descrip¬ 
tion  (IDD)  which  must  be  made  available 
at  PDR  and  maintained  under  Class  II  con¬ 
trol  until  FCA/PCA.  These  functions  may 
be  managed  as  one  large  CPCI  or  as 
several  (four  or  more  in  the  example  of 
Figure  5.3-2).  Regardless  of  level  of 
CPCI  designation,  each  major  functional 
element  or  module  design  should  be 
governed  by  individual  - 

a.  Design  requirements 

b.  Design  description 

c.  External  and  internal  interface 
descriptions 

d.  Flow  diagrams 

e.  Narrative  description 

f.  Test  requirements 


The  CPDP  should  describe  the  type  of 
documentation  required  to  define  the 
above  information  for  each  module.  The 
foregoing  design  definition  effort  con¬ 
tinues  until  CDR,  at  which  time  coding 
and  debug  begins.  At  this  point  the  mod¬ 
ule  designs,  defined  in  released  engi¬ 
neering  documentation,  are  submitted  to 
SCM  for  contractor  internal  control 
within  the  CPL.  No  changes  can  be  made 
to  these  designs  without  authorization 
from  the  software  design  manager.  This 
proviso  insures  that  an  approved  design 
will  not  be  altered  by  programmers  sub¬ 
sequently  coding  to  that  design.  Figure 
5.3-3  depicts  the  levels  of  configura¬ 
tion  control  required  from  PDR  (allo¬ 
cated  baseline)  through  FCA/PCA  (product 
baseline).  It  is  seen  that  both  the  modu¬ 
lar  designs  and  the  code  evolve  through 
increasingly  tighter  levels  of  configura¬ 
tion  control.  Concurrent  with  code  and 
debug,  the  programmer  conducts  module 
tests  until  he  is  satisfied  that  his 
code  functions  in  accordance  with  the 
module  performance  requirements.  He  then 
submits  his  code  to  the  CPL  where  it  is 
"formally  configured"  for  the  first 
time.  It  is  given  a  configuration  number 
(see  paragraph  4.3)  and  stored  for 
integration  with  other  modules  to  form 
the  next  higher  module  or  CPCI  version. 
When  all  functional  modules  are  avail¬ 
able  for  a  given  CPCI  version,  they  are 
integrated  (assembled,  compiled,  and 
linked  under  support  software  control) 
to  form  the  first  CPCI  "version".  This 
"version"  is  then  subjected  to  inter¬ 
module  compatibility  tests,  modified  as 
required  under  SCM  controlled  conditions 
(further  discussed  in  paragraph  7.3), 
and  then  turned  over  for  formal  QA 
acceptance  testing.  At  this  time  no 
changes  can  be  made  to  the  source  code 
of  any  module  without  a  contractor 
committed  Class  II  change.  This  proviso 
insures  all  design  subgroups  potentially 
impacted  by  the  change  have  an  oppor¬ 
tunity  to  evaluate  impact.  Finally  after 
QA  acceptance  (to  approved  contractor 
internal  validation  procedures),  FCA/PCA 
is  held  to  approve  the  product  baseline 
at  which  time  Class  I  (customer  approv¬ 
al)  control  prevails. 


Figure  5.3-3.  Configuration  Control  Phasing 


5. 3. 2.1  "Block  Change"  Approach.  Fre¬ 
quently  changes  originating  from  the 
various  design  subgroups  are  so  volumi¬ 
nous  that  configuration  control  becomes 
difficult  and  unwieldy.  Literally  hun¬ 
dreds  of  "make  work"  type  code  modifi¬ 
cations  may  be  accumulated  over  several 
months  against  a  given  CPCI  version. 
Such  changes  can  be  incorporated  in  two 
ways. 

a.  Handload  or  "patch"  the  object  pro¬ 
gram 

b.  Recompile  the  modified  source  code 

There  are  undesirable  aspects  of  using 
either  method  exclusively.  "Patching" 
the  object  program  with  hundreds  of 
changes  causes  program  performance 
regression.  Recompiling  a  new  total 
program  to  incorporate  every  minor 
change  and  then  running  a  complete 
retest,  is  costly  and  inefficient. 

A  compromise  is  achieved  by  using  the 
"Block  change"  approach.  In  this  method 
changes  are  packaged  together  or 
"blocked"  in  accordance  with  some  ground 
rule  for  manageability.  Factors  deter¬ 
mining  block  size  might  be 

a.  Quality  of  changes 

b.  Schedule  -  critical  project  mile¬ 
stones 

c.  Major  redesign 

d.  Merge  of  two  or  more  CPCIs  into 
one 

e.  Other  technical /management  reasons 

Under  this  system,  all  changes  accumu¬ 
lated  under  "Block  1"  are  identified, 
tracked  and  controlled  by  SCM.  At  a  con¬ 


venient  point,  patching  of  the  Block  1 
CPCI  versions  is  terminated,  the  source 
program  is  updated  to  incorporate  the 
intent  of  all  Block  1  changes,  and  a  new 
"Block  2"  version  is  compiled  and  com¬ 
pletely  retested.  Each  successively 
developed  "Block"  may  require  several 
CPCI  versions  to  accommodate  a  "test- 
debug-patch  reconfigure-retest"  cycle. 
This  block  approach  is  depicted  schemati¬ 
cally  in  Figure  5.3-4.  Under  this 
approach  the  first  version  of  new 
"block"  is  a  new  recompilation  of  the 
original  or  updated  source  program.  It 
is  retested  in  total  -  to  verify  all  per¬ 
formance  requirements  are  met.  As  the 
design  matures,  changes  are  required  to 
accommodate  performance  improvements  or 
to  fix  errors.  Each  individual  change  or 
error  is  evaluated  and  a  "patch"  is 
designed  to  the  object  program.  The  code 
is  changed  and  that  portion  of  the  test 
requirements  affected  by  the  patch  is 
retested.  The  patch  is  logged  and  even¬ 
tually  a  corresponding  source  code 
change  is  made  and  "rolled  in"  at  the 
next  "Block". 

Under  this  approach,  software  perfor¬ 
mance  is  evaluated  on  a  system  level 
early  in  the  development  cycle  to 
uncover  major  system  level  design 
incompatibilites.  Each  version  performs 
slightly  better  than  the  previous.  Each 
"Block"  performs  better  than  the  pre¬ 
vious  Block,  until  all  Part  1  require¬ 
ments  are  met  or  exceeded. 

5. 3. 2. 2  Baseline  Control  Mechanisms  - 
Summary.  The  preceding  discussion 
describes  the  "concepts"  of  baseline  con¬ 
trol  and  the  variants  in  control  con¬ 
cepts  required  to  accommodate  inherent 
differences  in  ATE  and  TS  software.  The 
following  section  discusses  methods  of 
managing  the  mechanizing  change  control. 
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BLOCK  1  BLOCK  1  BLOCK  1 
VERSION  1  VERSION  2  VERSION  3 


Figure  5.3-4.  Block  Change  Approach  to  Change  Control 


Section  6.0  CHANGE  MANAGEMENT 


The  subject  of  software  change  manage¬ 
ment  embodies  the  preparation,  evalua¬ 
tion,  approval,  incorporation,  account¬ 
ing,  and  test  of  all  customer  and  con¬ 
tractor  initiated  changes  to  CPCI's. 
Change  management  encompasses  changes  in 
requirements,  hardware  and  software 
interfaces,  problem  and  errors,  enhance¬ 
ments,  etc.  It  includes  change  board 
activities,  change  reporting  and  sta- 
tusing,  problem  reporting,  tracking  and 
corrective  action.  Some  of  the  unique 
problems  of  change  management  for  ATE 
and  TS  software  are  discussed  in  this 
section. 

Change  management  embodies  the  following 
elements 

a.  Change  classification 

b.  Change  approval  and  release 

c.  Change  accountability  and  verifi¬ 
cation 

The  following  subsections  discuss  each 
of  these  elements  as  they  apply  to  ATE 
and  TS  software. 

6.1  CHANGE  CLASSIFICATION 

Formal  changes  to  CPCI's  are  classified 
as  Class  I  (design)  or  Class  II  (discrep¬ 
ancy)  in  accordance  with  MIL-STD-483, 
Appendix  XIV.  An  engineering  change  is 
classified  as  Class  I  when  it  affects 
the  contractually  specified  form,  fit, 
function,  cost  or  delivery  schedule  of  a 
CPCI.  An  engineering  change  is  classi¬ 
fied  as  Class  II  when  it  does  not  fall 
within  the  criteria  for  Class  I.  Once  an 
external  baseline  is  established  (see 
Figure  5.1-1),  all  changes  are  processed 
as  Class  I.  Every  request  for  change  is 
evaluated  by  configuration  management  to 
determine  classification  and  type.  The 
criteria  fcr  classification  is  as 
fol lows: 


6.1.1  Class  I  Change  Criteria 

All  proposed  engineering  changes  to 
CPCI's  are  designated  as  Class  I  changes 
if  one  or  more  of  the  following  are 
affected: 

a.  Contractual  specifications  (func¬ 
tional,  allocated  and  product  baselines) 

b.  Contractual  plans 

c.  Cost,  fees,  incentives 

d.  Key  milestone  schedules,  deliv- 
eri  es 

e.  Statement  of  work  (SOW)  and/or  the 
contract  terms  and  conditions 

f.  Government  furnished  equipment 
(GFE),  safety,  delivered  manuals 

6.1.2  Class  II  Change  Criteria 

All  changes  not  considered  to  be  Class  I 
are  designated  as  Class  II  when  they  are 
encountered  during  a  period  designated 
as  "contractor  committed  Class  II  change 
control." 

6.1.3  Change  Processing 

Formal  change  processing  requires  that 
the  problem  be  identified,  and  pro¬ 
cessing  initiated  on  the  appropriate 
change  paper.  The  proposed  change 
request  is  classified  as  Class  I  or 
Class  II  and  is  coordinated  with 
affected  organizations  to  determine  its 
scope  and  impact.  It  is  then  presented 
to  the  program  change  board  for  evalua¬ 
tion  of  the  impact  on  cost,  schedule, 
design,  etc.,  and  for  approval  or 
disapproval.  The  change  paper  and  method 
of  processing  formal  changes  is  depen¬ 
dent  upon  the  classification  of  the 
change.  This  is  described  as  follows: 

6. 1.3.1  Class  I  Changes.  Class  I 
changes  are  processed  as  either  engi¬ 
neering  change  proposals  (ECP's)  or  con- 
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tract  change  proposals  (CCP's).  They 
must  be  approved  by  the  customer  prior 
to  implementation. 

a.  An  ECP  is  used  to  propose  an  engi¬ 
neering  change  to  a  contractual  specifi¬ 
cation  or  an  approved  configuration 
identification/baseline.  The  ECP  is  a 
comprehensive  document  which  contains 
provisions  for  supplying  all  the  infor¬ 
mation  necessary  to  make  a  thorough  eval¬ 
uation  of  the  change  and  its  impact  on 
the  entire  system. 

b.  A  CCP  is  used  to  propose  a  non¬ 
engineering  change  to  the  contract 
requirements;  e.g. ,  changes  to  the  SOW, 
contractual  schedules,  contractual 
plans,  equipment  quantities,  costs,  etc. 

6. 1.3. 2  Class  II  Changes.  Class  II 
changes  are  processed  as  either  a 
committed  or  noncommitted  change.  Class 
II  committed  changes  are  identified  as 
committed  changes  and  Class  II  non¬ 
committed  changes  are  as  noncommitted 
changes,  or  liaison  changes.  Class  II 
committed  changes  may  be  implemented 
after  the  program  change  board  approval 
and  commitment  is  established,  without 
prior  customer  approval;  however,  they 
must  be  submitted  to  the  customer  for 
concurrence  with  the  classification. 
Class  II  noncommitted  changes  do  not 
require  change  board  action  or  commit¬ 
ment. 

a.  A  committed  change  is  any  Class  II 
change  requiring  interorganization  coor¬ 
dination  through  the  program  change 
board  for  scheduling  and  release  commit¬ 
ment.  Committed  changes  are  serially 
numbered. 

b.  Liaison  (noncommitted  Class  II): 
This  type  of  Class  II  change  is  made  to 
permit  software  conformance  to  the 
intended  design.  It  is  fully  coordinated 
by  the  design  engineer  with  the  liaison 
engineering,  quality  assurance  and  manu¬ 
facturing  planning  organizations. 


6.1.4  Software  Change  Initiation 

During  the  software  development  process, 
design  deficiencies  or  coding  errors  may 
be  discovered  and  a  change  to  the  soft¬ 
ware  (design  documentation  or  code)  may 
be  required.  These  deficiencies  or 
errors  must  be  documented  and  reported 
on  appropriate  forms  so  that  the  problem 
may  be  analyzed  and  appropriate  action 
taken. 

The  type  of  form  used  to  report  these 
problems  and  the  type  of  change  control 
required  depends  on  the  stage  of  soft¬ 
ware  development,  whether  an  external  or 
internal  baseline  is  affected  and 
whether  the  deficiency  or  error  is  a 
design  error  or  coding  error. 

Irrespective  of  the  phase  of  develop¬ 
ment,  all  changes  precipitated  by  soft¬ 
ware  errors  should  be  reported  on  a  soft¬ 
ware  problem  report  (SPR),  (see  Figure 
6.1-1).  This  form  provides  for  orderly 
definition,  solution  and  tracking  of  all 
software  errors.  It  is  also  the  prime 
vehicle  for  analyzing  trends  in  software 
discrepancies. 

Design  changes  not  due  to  errors  (perfor¬ 
mance  improvements,  requirements 
changes)  are  described  and  authorized  on 
a  Design  Change  Request  (DCR)  or  equiva¬ 
lent  -  (Figure  6.1-2).  These  forms  are 
internally  controlled  by  the  Software 
Design  and  SCM  organization.  During  for¬ 
mal  change  control  periods  (Figure 
5.3-3),  they  do  not  in  themselves  autho¬ 
rize  software  drawing  changes,  but 
rather  define  the  reason  for  and  descrip¬ 
tion  of  an  actual  software  change.  The 
engineering  drawing  or  document  is 
changed  only  by  committed  change. 
Changes  originating  from  either  errors 
or  design  change  are  evaluated  and  autho¬ 
rized  by  change  board  activity  to  be  dis¬ 
cussed  next. 

6.2  CHANGE  APPROVAL  AND  RELEASE 

In  organizing  for  management  of  complex 
software  controlled  acquisition  systems, 
a  contractor  may  establish  two  change 


SOFTWARE  PROBLEM  REPORT 


Project  Name _ Computer/Lab  Utilized 


Problem  Report  No. 
Program  _ 


Problem  Discovery 


Name  of  finder 


Method  of  detection: 
□Usage 

□  Inspect!  on/ Ana  lysis 


□  Development  test 

□  Integration  test 
□Acceptance  Test 


Description  of  symptoms 


_ _ Date _ 

Tools  used  to  detect: 

□  None  □Dump  (terminal)  D Simulation 

□  Design  review  □Dump  (dynamic)  □  Assertion 

□  Peer  review  □  HOL  Debug  O  Proof 

□  Analyzer  □  Other 


Configuration  level  _ 

Correction  importance/need  date  _ 

Authorizing  Signature _ Orgn.  _ Date 


Problem  Analysis 
Findings  _ 


Name  of  analyst 


Start  date  _ End  date 


Resources  expended:  person _ computer  _ 

Estimated  resources  to  correct:  person  _ _ computer 


Problem  Correction 


Name  of  programmer 


Description  of  Correction 


Start  date _ End  date 


Components  changed  and  configuration  level  _ 

Resources  expended:  Person  _ computer _ 

Problem  category  - 

□  Job  control  language  DOesign  error  -  omitted  logic  □Documentation 

□  Operational  Interface  □Design  error  -  faulty  logic  DOther 

□  Coding  error  -  data  declaration  DTesting 

□  Coding  error  -  executable  instruction  ^Configuration  management 


Final  Authorizing  Siguature 


Orgn. 


Date 


Figure  6. 1 - 1.  Software  Problem  Report 
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DESIGN  CHANGE  REQUEST  (OCR) 


Report  No. _ 

Orl gl nator _ 

Organization  Date 

Reason  for  Change 


Proposed  Solution 


CI's/CPCI's  Impacted  Documents  Affected 


Cl  No.  Rev.  Version  •  IDD  /  / 

•  Part  I  Spec  /  / 

•  Part  II  Spec  /  / 

•  VDD  /  / 


•  Operating  &  Inst./  / 

•  Test  Procedures  /  / 


Change  Description 


Programmer 


Approval s 

•  Software  Design _ 

•  Quality  Assurance _ 

•  Systems  Eng.  _ 

•  Hardware  Design _ 

•  Test  and  Evaluation 


Figure  6. 1-2.  Design  Change  Request 


board  functions.  One  is  the  regular 
change  board  which  process  and  commits 
all  system  changes,  hardware  and  soft¬ 
ware.  In  addition,  in  order  to  manage 
the  large  volume  of  software  development 
changes  which  determine  the  evolution  of 
the  ultimate  operational  program,  a  soft¬ 
ware  change  board  may  be  established. 
Its  function  is  to  coordinate  proposed 
changes  in  software  design  among  pro¬ 
grammers,  hardware  designers,  interface 
specification  managers  and  systems  engi¬ 
neers  to  insure  the  compatibility, 
correctness  and  documentation  of  all  pro¬ 
gram  changes.  This  board  is  chaired  by 
the  SCM  group  which  insures  the  coor¬ 
dinated  approval,  documentation,  incor¬ 
poration  and  test  of  all  changes  are  con¬ 
trolled  and  recorded.  Module  level 
changes  to  CPCI's  not  yet  committed  to 
formal  release  are  normally  excluded, 
however,  once  released  CPU's  should 
require  this  level  of  coordination. 

6.2.1  Project  Change  Board 

The  project  change  board  meets  periodi¬ 
cally  to  act  on  proposed  Class  I  and 
Class  II  changes.  This  board  ensures 
that  changes  are  properly  classified; 
that  their  effect  on  interfacing  engi¬ 
neering  disciplines,  cost  and  schedules 
are  properly  assessed;  and  approves  the 
implementation  of  Class  II  changes  and 
the  transmittal  of  proposed  Class  I 
changes  to  the  customer  for  approval  or 
disapproval.  SCM  is  a  member  of  this 
change  board.  SCM  will  support  this 
change  board  to  control  proposed  soft¬ 
ware  changes,  assure  completeness,  and 
assess  impact  on  documentation. 

6.2.2  Software  Change  Board 

A  software  change  board  or  software  con¬ 
figuration  control  board  is  established 
and  chaired  by  the  software  manager  or 
his  designee  to  provide  a  means  of  con¬ 
trolling  and  processing  internal  soft¬ 
ware  changes.  This  board  reviews  all 
software  group  originated  changes  and 
approves  or  disapproves  the  change 
before  it  reaches  the  project  change 
poard  (or  during  periods  of  internal 


control  only).  Board  members  are 
appointed  by  the  software  manager  and 
normally  include  software  design,  test, 
quality  assurance  and  systems  engi¬ 
neering  representatives.  The  software 
change  board  working  in  conjunction  with 
the  SCM  organization  is  responsible  for 
change  approval  and  release.  A  typical 
flow  for  internal  software  changes  is 
shown  in  Figure  6.2-1. 

When  change  control  progresses  into 
formal  Class  II  control  phases  (CDR  for 
design;  completion  of  Intermodule  Com¬ 
patibility  Tests  (1MCT)  for  coding,  see 
Figure  5.3-3)  the  software  change  board 
(SCB)  functions  continue  unchanged.  How¬ 
ever,  the  SCB  approved  change  must  then 
be  committed  and  scheduled  in  the  pro¬ 
ject  change  board.  The  project  change 
board  assures  the  necessary  drawings  and 
documents  are  changed  and  appropriate 
retesting  are  accomplished  to  implement 
the  SCB  submitted  change.  Once  all 
changes  are  initiated,  authorized  and 
scheduled  by  the  applicable  change  board 
they  are  incorporated  as  changes  to  the 
source  and  object  documentation  and 
media  (decks,  tapes,  listings).  This 
change  incorporation  and  verification  is 
called  configuration  accountability  and 
is  discussed  in  the  following  section. 

6.3  CHANGE  ACCOUNTABILITY  AND 
VERIFICATION 

Change  accountability  is  one  of  the  most 
essential  tasks  in  achieving  configura¬ 
tion  integrity  of  a  deliverable  CPCI. 
Unless  properly  managed  during  develop¬ 
ment  and  validation  testing,  configura¬ 
tion  control  can  be  easily  lost.  This 
danger  arises  primarily  because  the  pro¬ 
gram  submitted  for  the  start  of  the  vali¬ 
dation  (or  acceptance)  testing  is  fre¬ 
quently  significantly  different  from  the 
program  which  completes  testing.  For 
example,  in  TS  software,  the  version  sub¬ 
mitted  for  validation  (system  level 
tests)  is  normally  the  version  com¬ 
pleting  verification  (single  thread. 
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Figure  6.2- 1.  Internal  Software  Change  Control 
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IMCT).  It  has  never  been  formally  exe¬ 
cuted  in  a  system  environment.  Under¬ 
standably  numerous  problems  arise.  As 
those  problems  increase  in  number 
throughout  the  validation  test  period 
(which  may  be  several  weeks  or  months) 
tremendous  tracking  and  control  problems 
can  arise.  For  example: 

a.  Test  conductors  may  change  with 
shift  change. 

b.  QA  personnel /technicians  change. 

c.  At  what  point  in  the  test  proce¬ 
dure  was  the  problem  encountered? 

d.  What  was  the  fix?  -  patch? 

recompiled  tape? 

e.  What  locations  and  data  were  modi¬ 
fied? 

f.  What  controls  insure  only  autho¬ 
rized  memory  locations  were  modified 
after  initial  load? 

g.  What  retest  was  done  — 

(1)  -  for  the  portion  of  the  pro¬ 
gram  failing? 

(2)  -  for  possible  regression  of 
the  program  in  other  areas? 

h.  At  what  point  in  the  procedure  was 
official  testing  resumed? 

i.  Who  authorized  the  fix  and  test 
restart? 

j.  Has  the  impacted  released  engi¬ 
neering  been  changed? 

Unless  rigorous  control  is  exercised  by 
QA  personnel,  any  of  the  above  factors 
can  create  a  configuration  loss.  It 
would  be  ideal  to  run  all  validation 
tests  so  that  official  "sell-off"  would 
be  a  smooth  formality  for  QA  and  the 
customer.  Unfortunately  schedules  don't 
always  allow  this.  The  result,  if  uncon¬ 
trolled  testing  prevails,  is  software 
which  is  inadequately  tested  or  ques¬ 


tionably  configured.  We  shall  examine 
methods  of  achieving  the  required 
accountability. 

Configuration  accountability  during  the 
pre-formal  test  phases  is  accomplished 
by  the  programmer  through  his  notes  and 
engineering  master  tapes  and  listings. 
When  modules  are  functionally  complete 
(module  verification  test  (MVT)  com¬ 
plete),  the  source  program  listing  (or 
symbolic  deck)  is  submitted  to  the  CPL 
for  control  by  SCM  personnel.  This  deck 
is  the  source  program  for  that  module; 
together  with  the  narrative  description 
and  flow  diagram  its  configuration 
definition  is  complete  up  to  that  point. 
Between  the  completion  of  MVT  and  start 
of  software  validation  tests  (SVT),  CPL 
personnel  must  account  for  the  configura¬ 
tion  of  that  module  and  all  like  mod¬ 
ules.  Upon  completion  of  Intermodule  Com¬ 
patibility  Tests  (IMCT),  the  primary 
source  of  further  changes  is  SVT.  This 
is  the  formal  acceptance  test  of  the 
software  for  contractor  QA  personnel. 
(See  "Software  Quality  Assurance"  guide¬ 
book,  section  6  for  a  discussion  of  for¬ 
mal  tests.)  Incorporation  of  "hand 
patched"  code  changes  are  physically 
verified  by  QA  via  one  of  the  following 
techniques. 

a.  Use  of  card  reader 

b.  Use  of  key  board  with  CRT 

c.  Use  of  interactive  text  editor 

d.  Use  of  data  cassettes 

In  each  of  the  above  mechanisms,  QA  can 
verify  configuration  by  observing  data 
changes  entering  memory  at  designated 
addresses.  Upon  completion  of  testing 
the  final  configuration  of  the  program 
is  the  original  program  tape  plus  a  veri¬ 
fied  card  deck  representing  drawing 
authorized  handpatches.  Alternately,  the 
original  load  tape,  modified  by  patches 
and  resident  in  memory,  may  be  dumped  on 
to  another  blank  tape  to  configure  the 
accepted  program. 


47 


Under  this  mode  of  configuration 
accountability,  a  previously  validated 
(acceptance  tested)  version  of  a  CPC  I 
could  be  "patched"  and  only  require 
verification  of  the  changed  addresses 
and  interfaces.  When  a  newly  recompiled 
version  is  submitted  for  validation,  how¬ 
ever,  it  will  most  likely  entail  a  com¬ 
plete  retest  since  the  effect  of 
recompilation,  assembly  edit,  etc.  on 
the  revised  source  code  demands  a  com¬ 
plete  re-evaluation  of  the  new  object 
program.  New  machine  code  is  no  longer 
physically  comparable  to  previously  vali¬ 
dated  code,  and  QA  must  regard  the  new 
program  as  an  unvalidated  baseline  or 
starting  point  for  controlling  a  new  set 
of  handpatches. 

For  ATE  programs,  many  of  the  automated 
configuration  control  aids  described  in 
paragraph  5.3.1  handle  the  task  of 
accounting  for  program  code  and  in  some 
cases  flow  diagram  changes.  When  used  in 
conjunction  with  authorized  procedures, 
their  use  simplifies  and  streamlines  pro¬ 
gram  configuration  accountability. 

Configuration  accountability  however, 
encompasses  more  than  just  code 
accountability.  Configuration  status 
accounting  documentation  is  the  means 
through  which  actions  affecting  CPCI's 
are  recorded  and  reported  to  program  and 
functional  managers.  It  principally 
records  the  "approved  configuration  base¬ 
line"  and  the  implementation  status  of 
changes  to  the  baseline.  In  this  way,  it 
provides  managers  with  confirmation  that 
change  decisions  are  being  implemented 
as  directed.  Configuration  status 
accounting  is  the  recording  and 


reporting  of  the  information  that  is 
needed  to  manage  configuration  effec¬ 
tively.  It  includes  listing  the  approved 
configuration  identification,  the  status 
of  proposed  changes  to  configurations, 
and  the  implementation  status  of 
approved  changes.  It  involves  main¬ 
taining  and  reporting  the  status  of  CPCI 
specifications,  associated  documents  and 
proposed  changes.  The  system  utilizes 
two  primary  reports;  the  Configuration 
Index  and  the  Change  Status  Report. 

The  computer  program  configuration  index 
provides  the  official  listing  of  the 
CPCI  specifications,  drawings  and  other 
significant  support  documents.  It  identi¬ 
fies  all  approved  changes  and  shows  the 
current  status  of  all  CPCI  documentation 
such  as  the  computer  program  development 
and  product  specifications,  test  plans/ 
procedures/reports,  handbooks,  manuals 
and  the  VDD.  The  change  status  report 
lists  all  proposed  changes  to  the  CPCI 
documentation  listed  in  the  configura¬ 
tion  index.  It  provides  information  on 
the  current  status  of  the  CPCI  and 
changes  throughout  its  development.  QA 
uses  the  configuration  index  and  change 
status  report  for  change  accountability 
to  assure  that  all  Class  I  changes  incor¬ 
porated  into  the  software  have  been 
approved  by  the  customer  and  that  no 
unapproved  Class  I  changes  are  incor¬ 
porated. 

The  guidebook  on  "Software  Quality  Assur¬ 
ance,"  Section  6.0,  describes  some  of 
the  contractor  internal  procedures  used 
to  achieve  software  configuration 
accountability  through  the  manufacturing 
record  system. 
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Section  7.0  COMPUTER  PROGRAM  LIBRARY 


The  Computer  Program  Library  (CPL)  func¬ 
tion  is  a  specific  requirement  of 
MIL-S-52779,  Software  Quality  Assurance 
Program  Requirements  and  is  treated  in 
depth  in  the  guidebook  on  Software 
Quality  Assurance.  However,  since  it  is 
a  vital  tool  in  maintaining  software 
configuration  control,  its  function  is 
briefly  summarized  herein. 

7.1  SOURCE  AND  OBJECT  CODE  CONTROL 

All  source  materials  used  to  configure  a 
CPCI  should  be  placed  under  internal  con¬ 
trol  after  MVT.  This  is  the  point  at 
which  the  programmer  has  declared  his 
module  to  be  satisfactory  for  integra¬ 
tion  with  other  modules.  It  should  be 
submitted  along  with  appropriate  decks, 
listings,  flow  diagrams,  descriptions, 
notes,  in  a  standard  format  for  integra¬ 
tion  with  other  like  modules.  Submission 
of  code  to  the  CPL  is  accompanied  by  the 
assignment  of  formal  software  component 
part  numbers  -  module  or  program  segment 
numbers  which  can  be  called  up  on  an 
engineering  configuration  drawing  such 
as  the  VDD.  It  is  ready  now  for  the 
first  phase  of  formal  control.  Unless 
changed  for  authorized  reasons,  it  is 
ready  for  compilation  and  assembly  into 
an  object  program. 

7.2  MEDIA  AND  DOCUMENTATION  SECURITY 

The  CPL  must  be  a  secure  repository 
which  protects  configured  media  and  docu¬ 
mentation  from  unauthorized  modifica¬ 
tion,  loss  or  damage.  Just  as  released 
engineering  drawing  masters  are  pro¬ 
tected  in  a  secure  vault,  programming 
records  require  similar  security.  It  is 
costly  to  reproduce  an  object  code  pro¬ 
gram  if  during  formal  testing  the  test 
copy  tape,  disk  etc.  is  accidentally 
damaged  or  lost.  If  the  source  materials 
used  to  build  the  program  master  tape, 
disk,  etc.  are  lost,  results  could  be 
catastrophic. 


Equipment  and  facilities  specifically 
designed  for  organized  and  secure  stor¬ 
age  of  computer  media  and  documentation 
are  available  commercially.  These  faci¬ 
lities  include: 

a.  Storage  cabinets  for  tapes,  disks, 
cassettes 

b.  Environmentally  designed  vaults 

c.  Mobile  cabinetry  for  listings, 
operating  manuals,  etc. 

d.  Color  coded  tape  seal  rings  for 
tape  categorization 

e.  Tape  disk  labels 

f.  Cypher  locked  cabinets  for  top 
security  items 

g.  Certified  magnetic  media 

This  available  equipment  can  be  pur¬ 
chased  by  modular  components  and 
arranged  by  simple  remove  and  replace 
operations  to  reconfigure  storage  faci¬ 
lities  to  meet  changing  needs.  A  good 
CPL  should  utilize  equipment  of  this 
sort  and  be  adequately  documented  to 
describe  location  and  access  require¬ 
ments  for  users. 

7.3  SUBMISSION  OF  SOFTWARE  FOR  TEST 

As  the  programmers  submit  completed  mod¬ 
ules  to  the  CPL  for  integration  into  a 
CPCI,  they  will  likely  be  in  various 
degrees  of  legibility,  completeness,  con¬ 
formance  to  prescribed  standards  and  con¬ 
ventions  for  programming  and  flow 
charting.  It  is  the  responsibility  of 
SCM  to  assure  either  within  the  library 
function  or  prior  to  submission,  that 
these  components  meet  prescribed  format 
requirements.  The  library  then  is 
responsible  for  tracking  and  assuring 
incorporation  of  all  approved  changes  to 
these  modules  or  components  and  corre¬ 
lating  them  to  requirements  for  the 
various  phases  of  testing.  For  example, 
it  is  probable  that  various  programs, 


representing  progressively  maturing  ver¬ 
sions  of  the  same  CPC  I,  may  be  developed 
in  parallel.  In  other  words.  Version  2 
of  CPC  I  X  may  be  just  beginning  test 
while  Version  1  of  the  same  CPCI  is  com¬ 
pleting  test.  Changes  applicable  to  each 
version  must  be  distinguished  if  changes 
have  different  version  effecti vities. 


It  is  a  CPL  responsibility  to  assure 
that  all  materials  needed  to  assemble 
the  deliverable  media  are  properly 
stored  and  maintained.  It  is  an  SCM 
responsibility  to  assure  that  those 
materials  are  properly  configured  by 
released  engineering.  The  integration  of 
responsibilities  for  defining,  building 
and  verifying  deliverable  media  is  shown 
in  Figure  7.3-1. 
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Figure  7.3- 1.  Integration  of  Media  Management  Responsibilities 


Section  8.0  REVIEWS  AND  AUDITS 


Formal  reviews  and  audits  are  the  pro¬ 
ject  milestones  which  formally  demarcate 
changes  in  development  phases  and  pro¬ 
vide  a  basis  for  product  acceptance  by 
the  customer.  These  reviews  and  audits 
are  essential  to  the  efficient  develop¬ 
ment  of  quality  software.  Design  reviews 
assess  the  completeness  and  evaluate  the 
results  of  major  development  phases 
before  proceeding  with  the  next  phase. 
Formal  audits  are  conducted  after  the 
software  system  testing  has  been  com¬ 
pleted  to  determine  whether  the  software 
and  supporting  documentation  meet  con¬ 
tract  and  project  requirements  prior  to 
system  acceptance.  The  guidebook  on 
"Reviews  and  Audit"  discusses  this 
subject  in  depth.  It  is  discussed 
synoptical  ly  here  since  it  is  an 
integral  part  of  configuration  manage¬ 
ment. 


8.1  DESIGN  REVIEWS 

8.1.1  System  Requirements  Review  (SRR) 

The  SRR  will  be  conducted  at  the  con¬ 
clusion  of  the  system  analysis  phase  to 
establish  the  adequacy  of  the  definition 
of  the  system  technical  requirements. 
The  entire  system  engineering  response 
to  the  SOW  and  specification  require¬ 
ments  will  be  reviewed.  The  software/ 
hardware  trade  studies  will  be  reviewed 
to  determine  that  they  are  sufficiently 
comprehensive  and  complete  for  allo¬ 
cating  functions  to  software.  The  allo¬ 
cated  software  functions  will  be  uni¬ 
quely  defined  and  not  be  dependent  on 
other  requirements.  Performance  limits 
will  be  designated  in  units  of  measure 
appropriate  to  software  and  will  be  veri¬ 
fiable.  This  review  will  cover  the  exter¬ 
nal  hardware  interfaces  and  dependen¬ 
cies. 


8.1.2  System  Design  Reviews  (SDR) 

SDR's  will  be  held  at  the  point  when  sys¬ 
tem  characteristics  have  been  defined 
and  functions  have  been  allocated  to  con¬ 
figuration  items, ’both  Cl's  and  CPCI's. 


These  reviews  will  evaluate  the  optimiza¬ 
tion,  correlation,  completeness,  and 
risks  associated  with  the  defined  allo¬ 
cation  of  requirements.  In  addition,  the 
engineering  process  that  produced  the 
allocation  will  be  reviewed. 

Software  requirements  will  be  reviewed 
for  technical  adequacy,  completeness, 
and  clarity.  Software  verification 
requirements  will  be  reviewed  for  com¬ 
pleteness,  adequacy  of  methods,  and 
traceability  and  compatibility  with 
higher  tier  specification  requirements. 
All  applicable  software  standards  for 
design  and  programming  will  be  identi¬ 
fied.  The  software  test  plan  will  be 
reviewed  to  ensure  that  it  satisfies  the 
test  requirements  of  the  development 
specifications. 

8.1.3  Preliminary  Design  Review  (PDR) 

The  PDR  will  be  conducted  at  the  con¬ 
clusion  of  the  preliminary  design  phase 
to  evaluate  progress  and  the  technical 
adequacy  of  the  basis  design  approach 
prior  to  the  detail  design  effort.  A  PDR 
will  be  conducted  for  each  CPC I. 

This  review  will  confirm  that  the 
initial  portion  of  the  computer  program 
product  specification  incorporates  and 
satisfies  all  requirements  in  the  com¬ 
puter  program  development  specification. 
The  allocation  of  CPCI  requirements  to 
individual  modules,  the  allocation  of 
storage  to  computer  programs,  timing 
estimates,  sequencing  requirements, 
operational  concepts,  and  data  base 
structure  and  organization  will  be 
reviewed.  All  functional  interfaces 
between  CPCI's  and  hardware  Cl's  will  be 
reviewed  for  consistency  and  compati¬ 
bility  with  interface  control  drawings 
and  basic  design  specification. 

8.1.4  Critical  Design  Review  (CDR) 

The  CDR  will  be  conducted  at  the  con¬ 
clusion  of  the  detail  design  phase  for 
each  CPCI  to  ensure  that  the  detail 
design  solutions  satisfy  the  performance 
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and  engineering  requirements  of  the  com¬ 
puter  program  development  specifica¬ 
tions.  This  review  will  establish  the 
integrity  of  the  computer  program  logi¬ 
cal  design  prior  to  coding  and  testing. 
The  review  will  establish  the  compati¬ 
bility  of  the  module  design  descriptions 
with  the  basic  design  description  and 
the  data  base  design  description.  All 
external  and  internal  interfaces  will  be 
reviewed  to  establish  system  compati¬ 
bility  of  design. 

The  software  test  plan  will  be  reviewed 
to  ensure  that  it  reflects  the  current 
information  developed  during  detail 
design.  All  computer  program  test  proce¬ 
dures  will  be  reviewed  for  compatibility 
with  the  design  and  test  requirements 
and  for  adequacy.  The  status  of  all 
changes  will  be  reviewed  to  ensure  that 
all  approved  changes  have  been  incor¬ 
porated  in  affected  documentation  and 
that  proposed  changes  have  been 
initiated  with  appropriate  change  paper. 

8.2  CONFIGURATION  AUDITS 

8.2.1  Functional  Configuration  Audit 
(FCA) 

The  FCA  is  a  formal  examination  of  the 
functional  characteristics  and  test  data 
for  each  CPCI  prior  to  acceptance.  The 
FCA  will  verify  that  the  CPC  1 1  s  actual 
performance  complies  with  the  perfor¬ 
mance  requirements  of  the  computer  pro¬ 
gram  development  specification.  The  test 
plans  and  procedures  and  the  test  data 
accumulated  during  software  testing  will 
be  reviewed  to  verify  that  the  item  has 
performed  as  required.  The  results  of 
analyses  or  simulations  for  those 
requirements  that  cannot  be  verified  by 
test  will  be  reviewed  to  ensure  their 
validity.  All  ECP's  and  PDR/CDR 


initiated  changes  will  be  reviewed  to 
ensure  that  they  have  been  incorporated 
and  verified.  A  list  of  all  documenta¬ 
tion  of  the  CPCI  will  be  reviewed  to 
ensure  adequate  documentation  of  the 
physical  configuration  for  which  test 
data  has  been  verified.  All  applicable 
computer  program  manuals  (user's,  pro¬ 
grammer's,  and  operator's)  will  be 
reviewed.  The  current  draft  of  the  com¬ 
puter  program  product  specification  will 
be  available  for  examination  to  provide 
guidance  for  conduct  of  the  PCA. 

8.2.2  Physical  Configuration  Audit 
(PCA) 

The  PCA  is  a  formal  examination  of  the 
"as-built"  configuration  of  each  CPCI 
prior  to  acceptance  and  delivery.  It  com¬ 
prises  an  examination  of  the  CPCI 
against  its  technical  documentation  to 
establish  the  CPCI's  product  configura¬ 
tion  identification.  The  PCA  will  eval¬ 
uate  the  adequacy  of  the  acceptance 
testing  requirements,  audit  the  engi¬ 
neering  drawings,  specifications,  tech¬ 
nical  data,  tests,  technical  descrip¬ 
tions,  flow  charts,  listings,  and  operat¬ 
ing  manuals.  The  VDD  will  be  examined  to 
ensure  completeness  of  descriptions  and 
records  of  change  control. 

The  completion  of  FCA/PCA  constitutes  a 
product  baseline.  All  changes  in  design 
or  product  configuration  must  be  pro¬ 
cessed  as  class  I  (customer  approval). 
In  most  cases  FCA/PCA  is  prerequisite  to 
product  acceptance  by  the  customer.  Ulti¬ 
mate  final  acceptance  is  contingent  upon 
delivery  installation  and  operational 
certification  by  the  customer. 
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Section  10.0  MATRIX:  GUIDEBOOK  TOPICS  VS.  GOVERNMENT  DOCUMENTATION 

The  elements  in  the  attached  matrix,  fig¬ 
ure  10.0-1  contain  Government  document 
sections  concerned  with  principal 
guidebook  topics. 
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Section  11.0  GLOSSARY  OF  TERMS 


I  1 


* 


Allocated  Baseline  -  The  approved  con- 
flguratlon  Item  Identification.  It 
governs  the  development  of  selected  con¬ 
figuration  items  that  are  part  of  a 
higher  level  specification,  e.g. ,  system 
specification.  It  Is  usually  defined  by 
the  Computer  Program  Development  Specifi¬ 
cation. 

Acquisition  Engineer  -  Military  or 
civilian  member  of  a  SPO  or  an  AFSC  divi¬ 
sion  who  supports  the  activities  of  a 
SPO. 

Baseline  -  An  authorized  documented  tech¬ 
nical  description  specifying  an  end 
item's  functional  and  physical  charac¬ 
teristics.  It  serves  as  the  basis  for 
configuration  control  and  status 
accounting.  It  establishes  an  approved 
well-defined  point  of  departure  for  con¬ 
trol  of  future  changes  to  system  or 
equipment. 


Control  Software  -  Software  used  during 
execution  of  a  test  program  which  con¬ 
trols  the  nontesting  operations  of  the 
ATE.  This  software  Is  used  to  execute  a 
test  procedure  but  does  not  contain  aqy 
of  the  stimuli  or  measurement  parameters 
used  in  testing  a  unit  under  test.  Where 
test  software  and  control  software  are 
combined  in  one  inseparable  program, 
that  program  will  be  treated  as  test 
software  (AFLC  66-37). 

Data  Base  -  A  collection  of  program 
code,  tables,  constants,  interface  ele¬ 
ments  and  other  data  essential  to  the 
operation  of  a  computer  program  or  soft¬ 
ware  subsystem. 

High  Order  Language  -  Problem  or  system 
oriented  code  which  can  be  automatically 
translated  to  machine  language  either 
directly  or  indirectly  (through  an 
assembly  language  step). 
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Certification  -  The  test  and  evaluation 
of  the  complete  computer  program  aimed 
at  ensuring  operational  effectiveness 
and  suitability  with  respect  to  mission 
requirements  under  operating  conditions. 

Computer  Program  Configuration  Item  -  A 
computer  program  or  aggregate  or  related 
computer  programs  designated  for  con¬ 
figuration  management.  A  CPC  I  may  be  a 
punched  deck  of  cards,  paper  or  magnetic 
tape  or  other  media  containing  a 
sequence  of  instructions  and  data  in  a 
form  suitable  for  insertion  in  a  digital 
computer. 

Configuration  Management  -  A  discipline 
applying  technical  ancT  administrative 
direction  and  surveillance  to: 

a.  Identify  and  document  the  func¬ 
tional  and  physical  characteristics  of  a 
configuration  item 

b.  Control  changes  to  those  charac¬ 
teristics;  and 

c.  Record  and  report  change  pro¬ 
cessing  and  implementation  status 


Host  Computer  -  An  off-line,  general  pur¬ 
pose,  programmable  computer  which  pre¬ 
pares  data  or  code  for  a  (target)  system 
computer,  e.g.:  ATE  central  computer. 

Product  Baseline  -  The  final  approved 
configuration  identification.  It  identi¬ 
fies  the  as  designed  and  functionally 
tested  computer  program  configuration. 
It  is  defined  by  the  Computer  Program 
Product  Specification  at  FCA/PCA. 

Software  Quality  Assurance  -  A  planned 
and  systematic  pattern  of  all  software  - 
related  actions  necessary  to  provide  ade¬ 
quate  confidence  that  computer  program 
configuration  items  or  products  conform 
to  establish  software  technical  require¬ 
ments  and  that  they  achieve  satisfactory 
performance. 

Software  -  A  combination  of  computer  pro¬ 
grams,  documentation  and  computer  data 
required  to  enable  the  computer  equip¬ 
ment  to  perform  computational  or  control 
functions  and  to  enable  program  main¬ 
tenance. 
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Software  Maintenance  -  Any  change  to  pre- 
viousiy  established  software.  Sources 
for  such  change  are  coding  errors, 
module  design  problems,  system  interface 
problems,  revised  system  requirements, 
and  capability  improvements. 

Support  Software  -  Auxiliary  software 
used  to  aid  in  preparing,  analyzing  and 
maintaining  other  software.  Support  soft¬ 
ware  is  never  used  during  the  execution 
of  a  test  program  on  a  tester,  although 
it  may  be  resident  either  on-line  or 
off-line.  Included  are  assemblies,  com¬ 
pilers,  translators,  loaders,  design 
aids,  test  aids,  etc.  (AFLC  66-37). 

System  Life  Cycle  -  The  system  acquisi¬ 
tion  life  cycle  consists  of  the 
following  five  major  phases  with  major 
decision  points: 

a.  Conceptual  phase 

b.  Validation  phase 

c.  Full-scale  development  phase 

d.  Production  phase 

e.  Deployment  phase 

(AFR-800-14,  Volume  II) 

Test  Software  -  Programs  which  implement 
documented  test  requirements.  There  is  a 
separate  test  program  written  for  each 
distinct  configuration  of  unit  under 
test  (AFLC  66-37). 

Top  Down  Structured  Programs  -  Struc¬ 
tured  programs  with  the  additional  char¬ 
acteristics  of  the  source  code  being 
logically,  but  not  necessarily  physi¬ 
cally,  segmented  in  a  hierarchial  manner 
and  only  dependent  on  code  already 
written.  Control  of  execution  between 
segments  is  restricted  to  transfer 
between  vertically  adjacent  hierarchial 
segments. 


Validation  -  System/software  validation 
7s  ffie  Tntegration  and  test  of  all 
hardware  and  software  components  to 
assure  the  complete  system  fulfills  all 
system  requirements.  Validation  is 
generally  regarded  as  the  act  of  exer¬ 
cising  the  software  after  verification 
testing  in  a  real  or  simulated  operating 
environment  against  a  precisely  defined 
set  of  mission  or  test  case  requirements 
deemed  representative  of  the  usage  envi¬ 
ronment  for  that  product.  Both  verifica¬ 
tion  and  validation  are  required  for 
software  acceptance. 

Verification  -  Computer  Program  Veri- 
fi cat  ion  is  the  iterative  process  of  con¬ 
tinuously  determining  whether  the  pro¬ 
duct  of  each  step  of  the  computer  pro¬ 
gram  acquisition  process  fulfills  all 
the  requirements  levied  by  the  previous 
step. 
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Section  12.0  ABBREVIATIONS  AND  ACRONYMS 


A/ I  -  Adapter/ Interface 

ATE  -  Automatic  Test  Equipment 

ATLAS  -  Abbreviated  Test  Language  for 
All  Systems 

CCP  -  Contract  Change  Proposal 

COR  -  Critical  Design  Review 

Cl  -  Configuration  Item 

CMP  -  Configuration  Management  Plan 

CPCI  -  Computer  Program  Configuration 
Item 

CPDP  -  Computer  Program  Development  Plan 

CPL  -  Computer  Program  Library 

CPPS  -  Computer  Program  Product  Spec. 

CPU  -  Central  Processor  Unit 

CRT  -  Cathode  Ray  Tube 

OCR  -  Design  Change  Request 

DID  -  Data  Item  Description 

ECP  -  Engineering  Change  Proposal 

FCA  -  Functional  Configuration  Audit 

GFE  -  Government  Furnished  Equipment 

GFP  -  Government  Furnished  Property 

HOL  -  Higher  Order  Language 

IDD  -  Interface  Design  Description 

IMCT  -  Intermodule  Compatibility  Test 

ITA  -  Interface  Test  Adapter 

LRU  -  Line  Replaceable  Unit 

MVT  -  Module  Verification  Test 

PCA  -  Physical  Configuration  Audit 


PDR  -  Preliminary  Design  Review 

PROM  -  Programmable  Read  Only  Memory 

QA  -  Quality  Assurance 

RFP  -  Request  for  Proposal 

SAE  -  Software  Acquisition  Engineering 

SCB  -  Software  Change  Board 

SCM  -  Software  Configuration  Management 

SDR  -  System  Design  Review 

SOW  -  Statement  of  Work 

SPO  -  System  Program  Office 

SPR  -  Software  Problem  Report 

SRR  -  System  Requirement  Review 

SVT  -  System  Validation  Test 

TPS  -  Test  Program  Set 

TRD  -  Test  Requirements  Document 

TS  -  Training  Simulator 

UUT  -  Unit  Under  Test 

VDD  -  Version  Description  Document 
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